Amazon CEO Andy Jassy: Cost of AI still more expensive than it should and will be

Key Concepts:

• AI Cost Optimization
• Data Center Demand
• DeepSeek LLM Impact
• AI Training vs. Inference
• Custom AI Chips
• Price Performance
• AWS Strategy
• Innovation Unleashing
• Tariffs

Data Center Demand and Growth:

• The speaker asserts that the growth in demand for data centers is so significant that there is no attenuation.
• They plan to continue building data centers to meet this high demand.
• The speaker states that even with potential tariff changes, they do not foresee attenuating their building plans for data centers.

Impact of DeepSeek and AI Efficiency:

• The emergence of DeepSeek, a new large language model (LLM) touted for its efficiency, initially raised questions about potentially reducing the need for extensive data centers and processing power.
• The speaker clarifies that DeepSeek's emergence did not change their forecasts for data center demand.
• The speaker expresses a desire to make AI less expensive for customers, acknowledging that current costs, driven by chips and other factors, are still higher than ideal.
• The speaker believes that lowering the cost of AI will enable companies to deploy it more expansively.
• The speaker draws a parallel to the early days of AWS, where reducing the cost per unit of compute and storage led to increased customer spending due to the ability to innovate and build more.

Cost Reduction in AI Compute:

• The speaker emphasizes the importance of reducing the cost of compute for AI.
• The speaker mentions that the AWS team views it as their responsibility and mission to make the cost of AI significantly lower than it is today.
• Two primary drivers for cost reduction are identified: chip costs and inference costs.
• The speaker notes that while most AI spending has historically been on training, the majority of spending shifts to inference (predictions) as AI applications scale into production.
• The speaker highlights the need to improve the price-performance of chips, which is a key reason why AWS is building its own custom AI chips.
• The speaker claims that their custom AI chips offer a 30% to 40% improvement in price-performance compared to existing GPU instances.
• The speaker emphasizes the importance of reducing the cost of inference, which involves various techniques.

AI Training vs. Inference:

• Training: The initial phase where AI models learn from large datasets. Historically, this has been the most expensive part of AI development.
• Inference: The process of using a trained AI model to make predictions or decisions on new data. As AI applications move into production, inference becomes the dominant cost driver.

AWS Strategy and Custom AI Chips:

• AWS is actively working to reduce the cost of AI compute, viewing it as a core mission.
• The development of custom AI chips is a key part of AWS's strategy to improve price-performance and lower costs.
• The speaker claims that AWS's custom chips offer a significant advantage in price-performance compared to existing GPU instances.

Conclusion:

The speaker emphasizes the continued growth in demand for data centers driven by AI, despite advancements in AI efficiency. A key focus is on reducing the cost of AI compute, particularly for inference, to enable broader adoption and innovation. AWS is pursuing this through custom AI chip development and other techniques, aiming for significant price-performance improvements. The speaker draws parallels to the early days of AWS, where cost reductions led to increased customer spending and innovation.