Are AI models running out of power? | The Economist

Key Concepts

• GPU (Graphics Processing Unit): Specialized electronic circuits designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device; in AI, these are the primary engines for training and running models.
• Inference: The process of running a trained AI model to respond to a user query.
• Tech Stack: The combination of technologies (hardware, software, networking) used to build and run AI applications.
• FABS (Fabrication Plants): Specialized manufacturing facilities where semiconductors (chips) are produced.
• Capex (Capital Expenditure): Funds used by a company to acquire, upgrade, and maintain physical assets such as property, buildings, or equipment.
• Lead Times: The amount of time that elapses between the initiation and completion of a process, specifically the delay in receiving hardware components.

---

1. The AI Supply-Demand Imbalance

AI companies are currently struggling to meet exponential demand, leading to service throttling.

• Anthropic: Modified terms of service to discourage usage during peak hours to manage capacity.
• OpenAI: Temporarily shut down its video generation tool, Sora, to redirect scarce computing resources toward more profitable ventures.
• Core Issue: Every user query requires "inference," which consumes significant processing power. As the user base grows, the demand for computing resources outpaces the available supply.

2. Infrastructure and Hardware Shortages

The industry is facing a bottleneck across the entire technology stack, not just in processors.

• Data Center Construction: The five largest US cloud providers (Amazon, Meta, Microsoft, etc.) are investing nearly $700 billion this year into AI data centers.
• Construction Delays: Projects face local opposition due to concerns over electricity, land, and water consumption.
• Component Shortages: There is a critical shortage of "old school" infrastructure, such as transformers and electrical switches, with lead times stretching between 3 to 5 years.

3. Semiconductor Choke Points

The hardware supply chain is constrained by two primary factors:

• Nvidia: Controls over two-thirds of the world’s AI processing power. Their chips are currently sold out, forcing some companies to utilize hardware that is two to three years old—a significant deficit in the fast-paced tech industry.
• TSMC (Taiwan Semiconductor Manufacturing Company): The primary manufacturer for most AI chips globally. Despite increasing capital expenditure by $60 billion, they cannot keep up with the industry's demand.

4. The Disconnect: Software vs. Hardware

A fundamental mismatch exists between the development cycles of software and hardware:

• Software: AI models improve and are replaced every few months.
• Hardware: Building the physical plants (FABS) required to increase chip capacity takes two to four years.

5. Industry Responses and Frustrations

• Sam Altman (OpenAI): Has publicly expressed frustration, urging TSMC to increase capacity.
• Elon Musk (Tesla/SpaceX): Proposed the "Terafab" project, an ambitious plan to build a fabrication plant by 2030 with more capacity than all current plants combined. Analysts estimate this would require between $5 trillion and $13 trillion in capital expenditure, highlighting the extreme scale of the current supply problem.

6. Economic Implications and Future Outlook

The supply crunch may force a shift in the AI business model:

• Pricing Pressure: Companies have been "burning cash" to keep inference costs low. If the supply crunch persists, firms will likely be forced to raise prices, which could slow down widespread AI adoption.
• Perspectives:
  • Some view the crunch as a "natural break" on reckless AI spending.
  • Others fear it will act as a significant barrier to the continued evolution and integration of AI technology.

Synthesis

The AI industry is currently trapped in a "build, build, build" cycle to address a massive supply-demand gap. While massive capital is being deployed, the industry is hindered by physical constraints—specifically the long lead times for data center infrastructure and the manufacturing limitations of semiconductor FABS. The fundamental disconnect between the rapid pace of software innovation and the slow, capital-intensive nature of hardware production suggests that the current "AI boom" may face a period of price hikes and slowed adoption as the physical world struggles to keep pace with digital demand.