Open Source Friday with Cilium

Key Concepts

• GitHub Copilot Spaces: A feature that provides context management for development workflows, allowing AI to understand and assist with code based on a repository's knowledge base.
• eBPF (extended Berkeley Packet Filter): A technology that allows sandboxed programs to run in the Linux kernel without changing kernel source code or loading kernel modules, enabling dynamic and safe extension of kernel functionality.
• Psyllium: An open-source networking, security, and observability solution built on eBPF technology, designed for cloud-native infrastructure, particularly Kubernetes.
• CNI (Container Network Interface): An interface for configuring network interfaces for Linux containers.
• Hubble: A sub-project of Psyllium that provides network observability by visualizing network flow logs and service maps, powered by eBPF.
• Tetragon: A Psyllium project that uses eBPF for runtime security and enforcement.
• Cluster Mesh: A Psyllium feature that allows seamless meshing of multiple Kubernetes clusters for benefits like failover and service discovery.
• Network Policy: Rules that control network traffic between pods and namespaces in Kubernetes. Psyllium extends Kubernetes' native network policies with Layer 7 capabilities and multi-cluster support.
• Zero Trust Networking: A security model that assumes no implicit trust and requires verification for every access request.
• eBPF Foundation: An organization focused on advancing the eBPF technology itself, supporting its development, security, and ecosystem growth.

GitHub Copilot Spaces: Context Management for Development

GitHub Copilot Spaces enhances development workflows by providing powerful context management. Users can create "spaces" by naming them and assigning them to individuals or organizations. These spaces can be populated with instructions that guide Copilot's assistance, including specific code updates and file modifications.

The core functionality of Copilot Spaces lies in its ability to ingest context from various sources:

• Repositories: Entire codebases or selected files can be attached.
• External Documents: Design specifications, ADRs (Architecture Decision Records), and performance reports can be included.
• Issues and Pull Requests: Linking these provides context on feature implementation and project history.

This rich context allows Copilot to provide expert guidance tailored to the specific project. For example, when asked to add a new pattern detection category for accessibility issues, Copilot can leverage the project's exact JSON format, severity levels, and testing patterns, offering advice grounded in the project's knowledge base rather than generic suggestions.

Copilot Spaces can be accessed directly within VS Code via the GitHub NCP servers Copilot Spaces toolset. Once a space is loaded, Copilot has access to all its context, enabling it to generate implementation plans that fit the existing architecture and stay synchronized in real-time with any updates to the space's content.

Psyllium: Cloud-Native Networking, Security, and Observability

Psyllium is presented as a critical solution for running Kubernetes in production, built on eBPF technology. It addresses the challenges of connecting and securing dynamic, ephemeral containerized environments at cloud-native scale.

Problem Solved: The transition from physical hardware to virtual machines and then to containers has made networking more complex. In distributed computing, the network is a critical component, but managing connectivity for thousands or millions of constantly spinning up and down containers is challenging. Psyllium provides reliable, fast, performant, and efficient networking for these environments.

Ubiquity in Kubernetes: Psyllium is widely adopted and often the default choice:

• Default CNI for Azure Kubernetes Service (AKS).
• Underlying fabric for Google Kubernetes Engine (GKE) Data Plane v2.
• Supported as a plugin on Amazon Web Services (AWS).
• Supported by hundreds of other cloud providers and major Kubernetes distributions.

eBPF as the Foundation: Psyllium's power stems from its foundation on eBPF.

Understanding eBPF: eBPF is described as a deep Linux kernel technology that allows for safe, sandboxed programs to run within the kernel. It addresses the challenge of balancing the stability of the Linux kernel (deployed on billions of devices) with the need for innovation and flexibility required by new technologies like cloud-native.

• Traditional Kernel Development: Adding new functionality to the Linux kernel traditionally involves a lengthy process of mailing list discussions, code reviews, and waiting for kernel releases, often taking years for innovations to reach users.
• eBPF's Approach: eBPF creates a sandbox environment within the kernel, allowing programs to be inserted and executed to change kernel functionality. This is compared to JavaScript's role in making web pages interactive.
• Benefits of eBPF:
  • Dynamic Functionality: Programs can be written, deployed, and executed without restarting the operating system, bringing new functionality to production systems in minutes or hours.
  • Security Enforcement: eBPF can intercept system calls and enforce security policies, such as blocking unauthorized file access.
  • Bug Mitigation: eBPF programs can be deployed to mitigate kernel bugs on-the-fly.
  • Innovation Cycle: eBPF significantly shortens the innovation cycle from years to hours or minutes.
  • Wide Adoption: eBPF is now a decade-old technology, and most modern kernels support it. It's used in various domains including networking, observability, runtime security, and profiling. Microsoft is also integrating eBPF into its products, with eBPF for Windows on the horizon.

Psyllium's Differentiators: While other Kubernetes networking solutions exist (Flannel, Calico, Weave, Civo OVN), Psyllium offers distinct advantages, primarily driven by its eBPF capabilities:

1. Networking:

   • Multi-Cluster Mesh: Psyllium enables seamless meshing of multiple Kubernetes clusters, providing benefits like failover, service discovery across clusters, and simplified cross-cluster communication. This makes it appear as a single, unified Kubernetes cluster.

2. Observability (Hubble):

   • Deep Visibility: Leveraging eBPF, Psyllium can see all network traffic within the Linux kernel.
   • Hubble Network Flow Logs: Provides detailed logs of network activity.
   • Service Map: Visualizes network communication flows, helping to debug complex distributed systems.
   • Self-Service for Developers: Enables developers to debug their own applications.
   • Case Study Example: YesNet (Energy Sciences Network) reported Hubble saving them days in debugging.

3. Security:

   • Advanced Network Policies: Psyllium extends Kubernetes' Layer 3/4 network policies with:
     • Layer 7 Policies: Allowing for more granular control based on application-layer protocols.
     • Multi-Cluster Policies: Enforcing policies across the cluster mesh.
     • Host Firewall: Applying network policies to the host itself.
   • Micro-segmentation: Enables fine-grained segmentation of network traffic.
   • Encryption of Network Traffic: Simplifies meeting regulatory requirements for data in transit encryption with a single switch.
   • Tetragon: A project for runtime security and enforcement, using eBPF to monitor and block suspicious activities within the kernel with low overhead and fine-grained control.

Psyllium in Action: A Live Lab Demonstration

The demonstration showcases Psyllium's installation and network policy enforcement using a live Kubernetes cluster.

Installation: Psyllium can be installed using the psyllium install CLI command. The lab environment allows users to see Psyllium components coming online as the cluster initializes.

Network Security Example (Death Star Scenario):

• Setup: A microservices application is deployed, simulating the Death Star, TIE fighters (Empire ships), and X-wings (Rebel ships).
• Initial State: Both TIE fighters and X-wings can access the Death Star, representing a lack of security policy.
• Applying Network Policy: A YAML-defined network policy is applied to restrict Death Star access to "Empire ships only" and to allow only TCP on port 80.
• Result: After applying the policy, X-wings (rebels) are blocked from accessing the Death Star, demonstrating effective network segmentation and security.
• Label-Based Policy: The demonstration highlights Psyllium's shift from IP-based security to label-based policies, which are more suitable for dynamic cloud-native environments. Labels allow consistent policy enforcement even as container identities or IP addresses change.

Observability with Hubble:

• Service Map Visualization: The Hubble UI is used to visualize the network policies in action. It shows that Empire ships can communicate within the cluster and with external pods, while external entities cannot access the Empire.
• Packet Inspection: Hubble inspects all packets, providing flow logs and reasons for dropped packets.

Psyllium's Evolution and Business Value

Psyllium has evolved beyond its initial role as a CNI plugin to address a broader range of cloud-native networking challenges, including:

• Layer 4 load balancing
• BGP for on-premise data centers
• Multi-cluster meshing
• Service mesh
• Host firewall
• Ingress

Many organizations are choosing Psyllium as a consolidated stack to solve multiple networking challenges, reducing the need for multiple point solutions.

Business Value Proposition: Psyllium addresses key business problems:

• Zero Trust Networking: Enhancing security by enforcing strict access controls.
• Network Automation: Reducing toil and overhead in dynamic cloud-native systems.
• Cost Savings: Optimizing compute and potentially carbon footprint.
• Multi-Cloud Connectivity: Facilitating seamless connections across different cloud environments.
• Failover Strategies: Ensuring business continuity.
• Tool Consolidation: Simplifying the IT stack by replacing multiple networking tools (e.g., cube proxy, ingress controllers) with a single solution.

Industry Examples:

• YesNet: Used Hubble for significant debugging time savings.
• Bloomberg: Leveraged Psyllium for securing financial data.
• Cloud Provider (Berlin-based): Replaced multiple networking tools with Psyllium, reducing complexity.

Operational Costs and Team Competency

• Cost: Running a networking solution in Kubernetes is necessary. Psyllium is often included by default in managed Kubernetes services like AKS and GKE.
• Migration: Psyllium offers migration labs and tools to ease the transition from other CNIs.
• Incremental Adoption: Users can adopt Psyllium incrementally, starting with network observability (Hubble) while keeping their existing CNI, and then gradually transitioning to Psyllium for routing.
• Team Competency:
  • NetOps Teams: For bare-metal data centers with complex networking (BGP, FRR), deeper expertise is required. However, much of this expertise translates to the cloud-native world, with a focus on learning new terminology.
  • Cloud-Native Teams: For those in the cloud, Psyllium can "just work" for basic networking, allowing them to focus on other aspects. The level of required expertise depends on how deeply one needs to engage with networking intricacies.

Contributing to Psyllium and the eBPF Ecosystem

Psyllium Project Involvement:

• Psyllium is the third fastest-moving project in the CNCF, with over a thousand contributors.
• Contribution Advice:
  • Pursue Passion: Focus on what interests you.
  • Solve Your Own Problems: Identify issues you encounter and contribute to fixing them. This intrinsic motivation is key to sustained contribution.
  • Start Small: Fix broken links in documentation or address specific feature gaps relevant to your use case.
• Resources: The Psyllium website, GitHub repository, and community Slack are available for engagement.

eBPF Foundation:

• Mission: To make eBPF the best technology for building infrastructure software, fostering its adoption and advancement.
• Focus: Unlike CNCF, the eBPF Foundation is dedicated to the upstream eBPF project itself.
• Key Activities:
  • Community Building: Funding developer conferences (e.g., Linux Plumbers, BPFConf) to bring people together.
  • Ecosystem Support: Funding projects that fall between individual company interests but benefit the broader ecosystem, such as improving kernel testing, architecture support, and program sanitization.
  • Security: Funding security audits and threat modeling for eBPF.
  • Academic Research: Funding PhDs to explore new use cases and advance eBPF capabilities.
  • Marketing and Outreach: Producing documentaries (e.g., "eBPF: The Invisible API"), white papers, case studies, and community fellowships to promote eBPF.
• Ecosystem Growth: The foundation has identified 78 companies with eBPF-based products, highlighting the technology's rapid expansion.

Bill Mulligan's Role: Bill Mulligan is a maintainer of Psyllium and a member of the eBPF Foundation governing board. He is active on LinkedIn.

Conclusion and Next Steps

The discussion highlights the transformative power of eBPF in cloud-native infrastructure, with Psyllium being a leading example of its application in networking, security, and observability. GitHub Copilot Spaces offers a complementary approach to enhancing developer productivity through AI-powered context management.

Call to Action:

• Try the Psyllium "getting started" lab and earn a badge.
• Star the Psyllium GitHub repository.
• Explore the eBPF Foundation's work and resources, including the eBPF documentary.
• Connect with Bill Mulligan on LinkedIn.
• Join the Psyllium community Slack.