A year in, Google wants its Axion processors to feel like a scheduling decision

Key Concepts

• ARM Neoverse: An architecture optimized for cloud-native workloads, offering higher energy efficiency and price-performance compared to traditional x86 processors.
• Google Axion: Google’s custom ARM-based processor designed for high-performance, energy-efficient cloud computing.
• Multi-arch Containers: Container images that contain binaries for multiple instruction sets (x86 and ARM), allowing them to run seamlessly on different hardware architectures.
• GKE Compute Classes: A Kubernetes Custom Resource Definition (CRD) that allows users to define priority lists for VM shapes, enabling dynamic provisioning and automated fallback mechanisms.
• Dynamic Resource Allocation (DRA): A Kubernetes API that decouples hardware implementation from workload requirements, allowing for more expressive and flexible resource requests (e.g., specific GPU capabilities).
• Tokens per Watt: A emerging metric for measuring efficiency in the AI era, shifting the focus from raw CPU speed to energy-constrained performance.

1. The Shift to ARM and Axion Processors

The discussion highlights a significant industry transition from traditional x86 server architectures to ARM-based processors.

• Performance and Efficiency: ARM processors, originally designed for mobile, have matured into high-performance server-grade hardware. They offer up to a 40% improvement in price-performance compared to x86.
• Energy Constraints: As data centers face power limitations, the industry is moving toward architectures that provide more "tokens per watt." The speakers argue that energy efficiency is no longer just an environmental goal but a financial necessity.

2. Kubernetes and Hardware Abstraction

The speakers emphasize that for most developers, the underlying hardware should be abstracted away by Kubernetes.

• The "Node is a Node" Philosophy: While hardware has become more specialized, Kubernetes acts as a universal control plane. If a container is built correctly, it should run on either architecture without code changes.
• Multi-arch Strategy: Developers are encouraged to build multi-arch containers. While building ARM images on x86 hardware can be slow due to emulation, moving the CI/CD pipeline to ARM-based instances can resolve this bottleneck and improve build speeds.

3. Managing Complexity with GKE Compute Classes

To handle the diversity of modern cloud hardware, Google introduced Compute Classes.

• Methodology: Instead of manually managing static node pools for every VM shape, users define a CRD with a priority list of VM configurations.
• Real-world Application:
  • Spiky Workloads: Customers use compute classes to define a baseline of on-demand VMs, with a fallback to spot instances of different configurations during traffic spikes.
  • Accelerator Availability: This framework helps manage the scarcity of GPUs by allowing workloads to automatically shift to available hardware configurations during training or fine-tuning jobs.

4. AI Workloads and Resource Management

AI is driving new requirements for infrastructure, particularly regarding data and hardware.

• Dynamic Resource Allocation (DRA): This API allows Kubernetes to treat specialized hardware (like GPUs) with the same flexibility as storage, enabling workloads to request specific capabilities (e.g., partial GPU memory) rather than just "a GPU."
• Data Handling: With AI models reaching terabyte sizes, traditional image downloading is insufficient. Technologies like image streaming and mounting container images as volumes are becoming critical to reduce startup times.

5. Key Arguments and Perspectives

• Low-Risk Migration: The speakers argue that moving to ARM is a low-risk experiment. Because Kubernetes allows for canary deployments (e.g., shifting 5% of traffic to ARM nodes), organizations can test performance and stability with an easy rollback path.
• Edge Cases: While most applications work seamlessly, developers should watch for "intricate edge cases," such as differences in floating-point math or highly optimized low-level database/caching systems.
• Security: The speakers maintain that security remains a constant concern regardless of architecture. They note that hardware-based vulnerabilities (like those seen in CPUs) prove that security is not purely a software problem.

6. Notable Quotes

• "We will end up selling watts, not CPUs and not tokens." — Highlighting the future of energy-constrained cloud computing.
• "The experiment is so cheap and the payoff can be so huge that it's worth just giving it a shot." — Regarding the transition from x86 to ARM.
• "If the security person says don't worry about it, they're not the security person you want." — Emphasizing that architecture changes do not exempt teams from rigorous security practices.

Synthesis and Conclusion

The transition to ARM-based processors like Google Axion represents a fundamental shift in how cloud infrastructure is consumed. By leveraging Kubernetes abstractions—specifically multi-arch containers and GKE compute classes—organizations can achieve significant cost savings and performance gains with minimal operational friction. The future of cloud computing is increasingly defined by energy efficiency ("tokens per watt") and the ability to dynamically manage heterogeneous hardware, ensuring that infrastructure can scale to meet the intense demands of modern AI workloads.