Do All Your AI Workloads Actually Require Expensive GPUs?

Google Axion and its Impact on Cloud Workloads: A Deep Dive

Key Concepts:

• Google Axion: Google’s ARM-based processor designed for cloud workloads, focusing on price-performance and energy efficiency.
• ARM Neoverse: The server-class CPU architecture underlying Axion, emphasizing energy efficiency.
• Custom Machine Types (M4A): GKE feature allowing users to define specific vCPU and memory configurations for VMs, optimizing resource allocation.
• Tranium: Google’s dedicated hardware for network and storage offloading, enhancing CPU performance.
• Multi-Architecture Compliance: Developing applications to run seamlessly on both x86 and ARM architectures.
• FinOps: Practices for optimizing cloud spending and resource utilization.
• Node Autoprovisioning (GKE): GKE’s ability to automatically provision nodes with optimal configurations based on workload demands.
• Hyperdisk: Google Cloud’s scalable block storage solution with independent IOPS and capacity scaling.
• Hyperdisk Pool: Enables fin provisioning for storage, optimizing costs for workloads with variable data usage.

1. The Context: Rising Workload Demands and the Need for Innovation

The discussion began by framing the emergence of Google Axion within the context of dramatically increasing workload demands in cloud services over the past decade. Alongside the growth of AI, there’s a need for more efficient and cost-effective infrastructure. This has spurred innovation in model deployment, development, and management. The shift towards ARM-based processors is presented as a key response to these challenges.

2. The ARM Story and Axion’s Foundation

Pne Bakray (ARM) explained that ARM Neoverse is the server-class CPU architecture powering Axion. ARM’s strategy focuses on establishing itself as a leader in server CPUs, leveraging inherent energy efficiency. This efficiency translates to benefits for developers deploying numerous applications, as ARM architecture supports high application density with reduced power consumption. The goal is to provide more compute for less cost.

3. Google’s History with Custom Silicon and the Axion Evolution

Andre (Google Cloud) detailed Google’s long history of custom silicon, starting with TPUs for machine learning. The development of Axion was driven by customer demand for increased performance at lower cost and power consumption. Google’s initial foray into ARM with the T2A machine served as an experiment, paving the way for the broader adoption of ARM with C4A and now N4A instances. N4A offers improved price-performance compared to C4A.

4. M4A and the Power of Custom Machine Shapes

Gary Singh (Google Kubernetes Engine) highlighted the significance of custom machine types (M4A) within the Axion ecosystem. M4A allows users to define precise vCPU and memory configurations, moving away from fixed-size VMs. This flexibility is crucial for optimizing resource utilization and cost, particularly for microservices and general-purpose workloads. This aligns with the growing importance of FinOps and centralized platform optimization.

5. Platform Engineering and Cost Optimization

The discussion connected Axion to the platform engineering trend. Platform engineering initially focuses on developer experience, but matures to encompass cost optimization and price-performance management. Axion’s superior price-performance provides a strong foundation for building efficient and cost-effective platforms.

6. Axion’s Technical Advantages: Efficiency and Performance

Axion’s benefits stem from its ARM architecture, which is inherently energy-efficient. Specifically, the ability to get a dedicated physical core per workload (unlike hyperthreading in some x86 architectures) provides more consistent performance. The C4A series is optimized for high throughput and low latency, while the N4A series is geared towards general-purpose compute. Andre mentioned specific instruction set optimizations (e.g., SVG) that enhance performance for certain workloads.

7. Tranium and Network/Storage Offloading

A key innovation in the C4A and N4A instances is Tranium, Google’s dedicated hardware for offloading network and storage tasks. This frees up CPU cycles for application processing, resulting in improved performance and scalability. Hyperdisk and Hyperdisk Pool further enhance storage performance and cost efficiency through independent IOPS and capacity scaling, and fin provisioning.

8. Price-Performance Gains and Real-World Examples

Concrete examples of price-performance improvements were provided:

• EngineX (Web Servers): 90% improvement in price-performance with M4A.
• Java Workloads: 80-85% improvement in price-performance.
• ZoomInfo: 60% improvement in price-performance for data analytics workloads.
• Data Analytics/ETL Pipelines: Significant performance gains with C4A and N4A, benefiting from ARM’s CPU-heavy nature.

9. AI/ML Inferencing with Axion

Axion is well-suited for AI/ML inferencing, particularly for smaller models that don’t require the full power of a GPU. It’s also effective for pre-processing tasks in larger AI pipelines. Google’s experience running LLMs on ARM-based processors in Pixel phones and through MediaPipe demonstrates the viability of this approach.

10. Multi-Architecture Deployment and Ease of Transition

The speakers emphasized the relative ease of deploying applications to both x86 and ARM architectures. Modern build tools and languages (e.g., Go, Python) simplify cross-compilation. Google has invested in tools and resources to facilitate the transition to ARM.

11. GKE Integration and Automation

Gary highlighted GKE’s integration with Axion, including node autoprovisioning and compute classes. These features automate the process of selecting and provisioning optimal node configurations based on workload requirements. GKE can automatically schedule ARM-targeted workloads onto appropriate nodes.

12. Edge AI and ARM’s Role

While primarily focused on cloud data centers, Axion’s underlying ARM architecture is also relevant to edge AI deployments. However, edge deployments require model quantization and consideration of form factor constraints. Google’s Android platform and MediaPipe framework are examples of ARM-based AI solutions at the edge.

Conclusion:

Google Axion represents a significant step towards more efficient and cost-effective cloud computing. By leveraging the benefits of ARM architecture, coupled with innovations like Tranium and custom machine types, Axion delivers substantial price-performance gains for a wide range of workloads, including general-purpose applications, data analytics, and AI/ML inferencing. The ease of multi-architecture deployment and GKE’s automation features further simplify the adoption of Axion for developers and platform engineers. Google’s commitment to ARM, both internally and through its cloud offerings, positions it as a key player in the future of cloud infrastructure.