CoreWeave CEO: Inference More Than 50% of AI Workloads

Key Concepts

• Systemic Structural Shortage: A global shortage of computing infrastructure, particularly in parallelized computing.
• AI Training vs. Inference: Training involves creating new AI models, while inference uses existing models to make predictions or decisions.
• Powered Shell: The physical infrastructure required to support a supercomputer, including the building, cooling systems, and electricity distribution.
• CapEx (Capital Expenditure): Investments in fixed assets like data centers and equipment.
• Planetary Scale Buildout: A massive, global expansion of computing infrastructure to support AI.
• Aging GPU: The process of older generations of GPUs being used for less demanding tasks like inference as newer generations are released.
• Long-Term Contracts: CoreWeave's business model relies on securing long-term contracts for compute capacity.
• Cost of Capital: The cost of obtaining financing for building out infrastructure.

Demand Outpacing Supply

The core issue is that demand for computing infrastructure is significantly exceeding supply, particularly for parallelized computing needed for AI. This shortage is driven by the rapid growth in both AI training and inference.

Scaling Infrastructure

CoreWeave is actively working to scale its infrastructure to address the supply-demand imbalance. This involves:

• Partnering: Collaborating with partners across the supply chain to overcome bottlenecks in delivering infrastructure.
• Comprehensive Solutions: Delivering complete supercomputer solutions, as "97% of a supercomputer doesn't do anything."
• Addressing Bottlenecks: Focusing on key bottlenecks like the "powered shell" (building, cooling, electricity distribution).
• Anticipating Future Bottlenecks: Recognizing that addressing one bottleneck will likely reveal another (e.g., silicon or networking shortages).

CapEx and Systemic Imbalance

The increased CapEx in the sector is primarily driven by the "planetary scale buildout" of computing infrastructure, not just incremental cost increases or supply chain issues. While there are challenges with specific components, the overall investment reflects the long-term need for AI computing power.

Training vs. Inference Demand

While demand for AI training remains strong, inference is experiencing a significant ramp-up. CoreWeave is seeing a shift in compute usage, with inference now accounting for over 50% of demand. Training jobs exhibit a step function pattern (high usage during training, then a drop), while inference shows a more gradual increase and decrease throughout the day.

GPU Aging and Utilization

Newer generations of GPUs (e.g., Blackwell) are primarily used for cutting-edge training, while older generations (e.g., H100, Ampere) are being utilized for inference. Demand for older GPUs remains strong, as they are absorbed into term contracts to support the growing inference needs.

Business Diversification

CoreWeave is actively diversifying its client base beyond major hyperscalers like OpenAI and Microsoft. They are penetrating additional layers of the enterprise space and seeing growth in new sectors integrating AI, such as:

• VFX (Visual Effects): Companies like Moon Valley are integrating AI into visual effects workflows.
• Life Sciences: Developments in AI for healthcare, such as Hippocratic A.I., are driving demand.

Collaboration vs. Going It Alone

While some companies like Meta (with Colossus) are building their own AI infrastructure, CoreWeave believes that partnerships are essential for scaling AI infrastructure effectively. They are working with Meta as a client and see opportunities to expand the relationship. The sheer scale and speed required for AI deployment necessitate collaboration.

Long-Term Contracts and Business Model

CoreWeave's business model is based on long-term contracts, which insulates them from short-term fluctuations in GPU spot prices. They secure contracts for massive compute capacity, then borrow money to build the infrastructure. This approach allows them to secure financing from sophisticated lenders.

Cost of Capital Reduction

A key focus for CoreWeave is reducing its cost of capital. They have made significant progress, achieving a 900 basis point decrease in their last delayed draw facility for non-investment grade clients. This improvement has a substantial positive impact on the company's economics.

Training and Inference Outlook

CoreWeave expects training demand to remain strong, but inference will likely dominate in the long term. They are well-positioned to supply inference capacity in a low-latency configuration through their network of 32 data centers.

Conclusion

The key takeaways are that the demand for AI compute is exploding, driven by both training and inference. CoreWeave is focused on scaling its infrastructure, diversifying its customer base, and optimizing its business model to capitalize on this growth. Their emphasis on long-term contracts and reducing the cost of capital positions them well to navigate the challenges and opportunities in the rapidly evolving AI landscape.