AI progress isn't linear, Leonis Capital partner says #AI #tech

Key Concepts

• Non-Linear AI Progress: The central idea that advancements in Artificial Intelligence don’t occur at a steady, predictable rate.
• Model Architecture: The fundamental design of an AI model, influencing its capabilities.
• Training Run: The process of feeding data to an AI model to improve its performance.
• GP4: (Likely referring to GPT-4) A specific, advanced large language model, used as a benchmark for progress.

---

The Non-Linear Nature of AI Advancement

The primary point emphasized is that observing and predicting AI progress requires understanding its fundamentally non-linear nature. Many business analysts incorrectly assume a consistent, linear trajectory of improvement. The speaker argues this is a flawed perspective. Instead, AI development proceeds in “lumps” – discrete, significant jumps in capability. These jumps aren’t gradual; they are triggered by specific events.

Catalysts for Significant Progress

These “lumps” of progress are specifically attributed to three key factors:

1. New Model Architectures: Innovations in the underlying design of AI models. This isn’t simply incremental improvement, but a fundamental shift in how the AI is built.
2. New Training Runs: Improvements stemming from utilizing different or larger datasets, or employing more effective training methodologies. The speaker highlights that simply doing a new training run, even with existing architecture, can yield substantial gains.
3. New Ways of Doing Things: This encompasses broader methodological breakthroughs in the field – new algorithms, techniques, or approaches to problem-solving within AI.

Addressing Perceptions of AI Progress – Speed and Stagnation

The speaker directly addresses the common, opposing viewpoints regarding the pace of AI development. They identify two prevalent narratives:

• Rapid Acceleration: The belief that AI is evolving at an incredibly fast rate, with dramatic differences between current capabilities and those of just two years ago.
• Perceived Stagnation: The assertion that AI progress has stalled, particularly since the release of GPT-4, with no significant advancements observed since.

The speaker refutes both extremes, stating that the reality lies “somewhere in between.” The fluctuating nature of progress – the “lumps” – explains why these conflicting perceptions arise. It’s not a consistent upward climb, so periods of rapid advancement are followed by periods where improvements seem less obvious.

Implications for Business Observers

The core takeaway is that business observers should avoid interpreting short-term fluctuations as indicative of long-term trends. Questions like “Why is AI progress slowing down?” or “Why is it happening so quickly?” are based on a misunderstanding of the underlying process. The speaker implicitly suggests that a more nuanced understanding of the non-linear nature of AI development is crucial for accurate forecasting and strategic planning.

Synthesis

The central message is a call for a more realistic and informed perspective on AI progress. Rather than expecting a steady, predictable rate of improvement, business observers should recognize that AI development occurs in bursts, driven by specific innovations in model architecture, training methodologies, and overall approaches. This understanding is vital for avoiding misinterpretations of short-term fluctuations and making sound strategic decisions.