Data center buildout is creating new winners, says BofA’s Andrew Obin

Data Center Infrastructure: Beyond the Chipmakers

Key Concepts:

• Data Center Infrastructure (Electrical & Cooling)
• Megawatt (MW) Capacity & Cost
• Nvidia's Accelerated Product Cycle & Rack Re-architecture
• Direct Current (DC) Architecture (400V-800V vs. 12V)
• Cooling Systems (HVAC)
• Electrical Systems
• Cost per Megawatt (Electrical & Cooling)
• Reliability & Redundancy

1. The Changing Landscape of Data Centers:

• The discussion centers on the evolving infrastructure needs of data centers, particularly concerning electrical and cooling systems.
• Historically, these systems were treated as commoditized components.
• Data centers are now significantly larger, averaging around 100MW compared to 10MW a decade ago.
• The cost per megawatt is approximately $50 million, making a 100MW data center a $5 billion enterprise.

2. Nvidia's Impact and Rack Re-architecture:

• Nvidia's accelerated product cycle is driving a fundamental re-architecture of the rack within data centers.
• This re-architecture necessitates a complete overhaul of existing electrical and cooling infrastructure.
• Traditional HVAC systems are becoming inadequate for the demands of new high-performance chips.
• Data centers are transitioning to direct current (DC) architectures, operating at higher voltages (400V-800V) compared to the previous 12V standard.

3. Electrical and Cooling Systems: From Afterthought to Critical Component:

• Previously, electrical and cooling systems were considered an afterthought in data center design.
• However, with Nvidia's advancements, these systems have become critical for the proper functioning of the entire data center.
• The Rubin architecture, for example, requires advanced cooling and electrical connections to operate effectively.

4. Cost Breakdown and the High Stakes:

• The average cost for a data center is estimated at $50 million per megawatt.
• Electrical systems account for approximately $1.9 million per megawatt.
• Cooling systems account for approximately $1.4 million per megawatt.
• Failure in either the electrical or cooling system can result in catastrophic damage to the expensive chips (approximately $40 million worth per megawatt), potentially burning them out within 30 seconds.

5. Key Arguments and Perspectives:

• The main argument is that the market is underestimating the importance and complexity of electrical and cooling infrastructure in modern data centers.
• The traditional approach of treating these systems as commodities is no longer viable.
• The increasing power density and heat generation of new chips require specialized and highly reliable infrastructure.

6. Notable Quotes:

• "Right now an average data center is around 100MW, right. The cost per megawatt is 50 million. So that's 5 billion enterprise."
• "You messed that up, right? $40 million worth of chips per megawatt pretty much burn within 30s."

7. Synthesis/Conclusion:

The key takeaway is that the data center landscape is undergoing a significant transformation driven by advancements in chip technology. Electrical and cooling infrastructure, once considered secondary, are now critical components that require careful planning and investment. Failure to address these infrastructure needs can result in significant financial losses due to equipment damage and downtime. The transition to DC architectures and advanced cooling solutions is essential for supporting the next generation of high-performance computing.