How Kubernetes Became the New Linux

Key Concepts

• Kubernetes (K8s): An open-source system for automating deployment, scaling, and management of containerized applications.
• Autoscaling: The ability of a system to automatically adjust its resource allocation (e.g., compute, memory) based on demand.
• CRD (Custom Resource Definition): A Kubernetes API extension that allows users to define their own custom resources and controllers.
• Controller: A Kubernetes component that watches for changes to custom resources and reconciles the actual state with the desired state.
• Microcontrollers: Small, specialized controllers designed to manage specific custom resources.
• Crow (Kubernetes Resource Orchestrator): An open-source project aiming to simplify resource orchestration within Kubernetes by automating CRD and controller creation.
• Crossplane: A Kubernetes add-on that enables managing cloud infrastructure through Kubernetes APIs.
• Carpenter: An open-source node autoscaler for Kubernetes that provisions nodes on demand based on scheduling pressure.
• SIG (Special Interest Group): A working group within the Kubernetes community focused on specific areas of development.
• CNCF (Cloud Native Computing Foundation): An organization that fosters and sustains an ecosystem of open-source, vendor-neutral projects to make cloud-native computing ubiquitous.
• DAG (Directed Acyclic Graph): A graph where nodes represent tasks or resources and edges represent dependencies, used to define execution order.
• Simple Schema: A YAML-based schema definition used by Crow to define CRDs and their constituent resources.
• Resource Graph Definition: A definition within Crow that specifies the CRD spec and the constituent resources to be managed.
• Provider Model: An architectural pattern in Carpenter that allows it to interact with different cloud providers (e.g., AWS EC2, Azure VM).
• Node Overlay: A feature in Carpenter that allows for more granular control over node selection criteria.
• Dynamic Capacity: Carpenter's default mode of provisioning nodes only when needed.
• Static Capacity: An option in Carpenter to pre-allocate compute resources.

Crow: Simplifying Kubernetes Resource Orchestration

1. The Need for Crow: The discussion highlights a historical challenge in Kubernetes: managing dynamic and spiky cloud-native workloads. Initially, users relied on static capacity, but the inherent nature of cloud-native applications demanded more dynamic resource management. Kubernetes' extensibility through CRDs and controllers offered a powerful solution, but building custom controllers for every orchestration need led to "controller proliferation" and specialized skill requirements. This resulted in organizations owning significant amounts of code that wasn't core business logic, essentially becoming technical debt.

2. Crow's Genesis and Philosophy: Crow emerged from AWS's experience operating Kubernetes at scale. The goal was to create a new "primitive" – a foundational element that the entire community could leverage, rather than a standalone project with its own identity. This approach aimed to avoid competing interests and integrate seamlessly into the Kubernetes ecosystem. Crow's core mission is to solve problems related to controller proliferation and resource orchestration.

3. Key Features and Technical Details of Crow:

• Automated CRD and Controller Creation: Crow automatically generates and registers CRDs and then creates a "microcontroller" to manage these custom resources.
• Simple Schema: This YAML-based schema allows users to define their CRD specifications.
• Resource Graph Definition: This single resource definition in Crow comprises two parts:
  • Spec: Defines the CRD itself.
  • Resources: Lists the constituent resources that make up the custom resource.
• Dependency Inference and DAG Creation: Crow analyzes the defined resources and their configurations to infer dependencies and create a Directed Acyclic Graph (DAG), dictating the order of operations.
• Kubernetes Native Principles: Crow operates using familiar Kubernetes principles (CRDs and controllers), making it accessible to platform engineers.

4. Current Status and Future: Crow is currently at version 0.6.61. A major feature for version management and rollouts is planned, after which the project is expected to be "code complete." While users are encouraged to use it with caution due to potential breaking changes, many are already running it in production. The project is seeing adoption from platform engineers building layers of abstraction.

Carpenter: Intelligent Node Autoscaling for Kubernetes

1. The Problem Addressed by Carpenter: Carpenter was conceived to address the complexities of Kubernetes autoscaling. The initial idea was to provision nodes on demand based on "scheduling pressure" – when a pod cannot be scheduled due to a lack of available nodes. This approach aimed to be a more dynamic and efficient alternative to existing autoscaling patterns.

2. Key Features and Technical Details of Carpenter:

• Just-in-Time Node Provisioning: Carpenter provisions nodes only when they are needed to satisfy pending pods.
• Deprovisioning: It also deprovisions nodes when workloads decrease, leading to cost savings.
• Intelligent Node Selection: Carpenter can select the "right node" based on performance requirements and price characteristics, further optimizing costs.
• Provider Model: This model allows Carpenter to integrate with various cloud providers and their specific APIs (e.g., EC2 for AWS).
• Elegant and Flexible API: The API design is simple yet allows for complex configurations. Users can request a basic node or specify detailed hardware characteristics (CPU, memory, specific hardware features).

3. Popularity and Adoption: Carpenter's popularity stems from its novel approach to autoscaling, its cost-saving potential, and its simplified operational burden. The elegant API design also contributes to its appeal. The project has gained traction and has been discussed with SIG Autoscaling.

4. Addressing Operational Challenges:

• API Stability: Carpenter has prioritized API stability since its 1.0 release, with significant upfront design to ensure future flexibility.
• Node Overlay: A backward-compatible enhancement that allows for overlaying node selection criteria, such as specific third-party pricing in cloud services.
• Capacity Options:
  • Dynamic Capacity: The default mode, provisioning nodes on demand.
  • Static Capacity: A new option for users who prefer pre-allocating compute resources for more simplicity or specific use cases.

5. Mission of Simplification: Both Crow and Carpenter align with a broader mission to simplify Kubernetes management. As Kubernetes adoption grows across all types of organizations, there's a need for standardized, scalable solutions. AWS aims to leverage its operational expertise to contribute to these standards and make Kubernetes more accessible.

AWS's Evolving Open Source Philosophy and Project Contributions

1. Shift in Open Source Strategy: The discussion highlights a significant evolution in AWS's approach to open source. Historically, large companies might have been hesitant to donate projects. However, there's now a clear trend towards contributing significant projects to foundations like the CNCF.

2. Rationale for Open Sourcing and Donation:

• Customer Needs: AWS's contributions are driven by a desire to solve customer problems. Projects like Carpenter and Crow originated from identifying and addressing specific pain points within the Kubernetes ecosystem.
• Community Focus: The cloud-native landscape is inherently community-driven. AWS recognizes that building solutions in isolation for their products is not valuable. True value comes from building in a way that benefits the broader Kubernetes community.
• Advancing Kubernetes: AWS aims to invest in open-source projects that will help Kubernetes evolve and move forward.
• Strategic Placement:
  • Carpenter: Donated to SIG Autoscaling, as it's considered a core component of an ideal Kubernetes experience.
  • Crow: Donated to SIG Cloud Provider, recognizing that resource orchestration problems often stem from managing cloud resource state within a cluster.

3. The "Primitive" Approach: AWS's strategy is to identify specific problems and develop "primitives" or features that can be integrated into existing ecosystems (like Kubernetes) rather than creating entirely new, competing products. This approach ensures broader adoption and impact.

4. Conclusion: The conversation emphasizes AWS's commitment to open source as a means to solve customer problems, foster community growth, and advance the Kubernetes ecosystem. Projects like Crow and Carpenter exemplify this philosophy by providing valuable, standardized solutions that simplify complex aspects of cloud-native development and operations. The goal is to contribute foundational elements that help Kubernetes continue to evolve and thrive for years to come.