Learning k8s - ep 6 - KCNA exam-prep app is live!

Key Concepts

• KCNA (Kubernetes and Cloud Native Associate): An entry-level certification focused on cloud-native technologies.
• Open Claw: An AI assistant project designed to run locally within a secure, monitored environment.
• Network Boundary: A security architecture using NGINX as a forward proxy to control and audit AI outbound traffic.
• OTEL (OpenTelemetry): A framework used for observability, specifically for capturing network request metadata.
• Sidecar Pattern: A design pattern where a helper container (like a proxy) is injected into a pod to handle tasks like traffic interception or encryption.
• Tokens per Watt: A metric used to evaluate the energy efficiency of hardware when running Large Language Models (LLMs).

---

1. Kubernetes Exam Preparation (KCNA)

Marco is expanding his open-source exam preparation tool to include the KCNA certification.

• Tool Features: The application is built using Python and the rich library for terminal UI. It includes 300 multiple-choice questions, detailed explanations, and direct links to official Kubernetes documentation.
• Methodology: The tool supports different modes: "Quick" (20 questions), "Full" (60 questions), and "All" (300 questions). It features randomized question ordering to ensure effective study.
• Technical Details: The questions are stored in a JSON file. The tool provides immediate feedback on incorrect answers, explaining concepts like Rolling Updates (the default deployment strategy) and the role of Envoy proxies in service meshes for mTLS (mutual TLS) enforcement.

2. AI Security: The "Open Claw" Network Boundary

Marco presented a proof-of-concept project aimed at securing AI assistants by controlling their network access.

• The Problem: AI agents often have unrestricted access to the internet, posing a security risk.
• The Solution: By running the AI model inside a Kubernetes environment, Marco uses NGINX as a forward proxy to enforce an "allow-list" of domains (e.g., duckduckgo.com, nginx.org).
• Implementation:
  1. Traffic Capture: Every request is intercepted by NGINX.
  2. Observability: Using OpenTelemetry (OTEL), the system generates records of all inbound and outbound traffic.
  3. Enforcement: If an AI agent attempts to access an unauthorized site (e.g., google.com), the system returns a 403 Forbidden status.
  4. Compliance: This approach allows for auditing network metadata without needing to decrypt encrypted traffic, maintaining privacy while ensuring security.

3. Hardware Efficiency for Local AI

Marco shared findings from his experiments on running LLMs (specifically the Qwen 2.5 3B model) on various hardware configurations.

• Key Finding: While high-end GPUs like the RTX 3090 provide high performance, they are less energy-efficient than mobile hardware for small models.
• Efficiency Metric: The RTX 3060 mobile (found in older laptops) demonstrated superior "tokens per watt" performance compared to desktop-class GPUs.
• Actionable Insight: For users looking to run local AI models cheaply, Marco suggests purchasing used laptops (even those with cracked screens) as a cost-effective and energy-efficient alternative to building high-end desktop rigs.

---

Notable Quotes

• "If the vendor is running their own model in a sandbox environment, I think we should too." — Marco, regarding the necessity of isolating AI assistants.
• "We're not decrypting any content... I'm just looking at the metadata connection metadata. So this means that even if you were to deploy this in an environment where you can't really look into encrypted traffic, you could just say, okay, we're not looking into what you're sending." — Explaining the compliance benefits of his network boundary design.

---

Synthesis

The session highlights a dual focus on Cloud Native education and AI infrastructure security. By leveraging Kubernetes for both exam preparation and as a secure runtime environment for AI, Marco demonstrates how standard DevOps tools (NGINX, OTEL, Kubernetes Network Policies) can be repurposed to solve modern AI security challenges. The transition from high-end enterprise hardware to repurposed mobile laptops for AI inference provides a practical, low-cost framework for developers to experiment with local models safely and efficiently.