Open Source Friday with Kong's OpenMeter

Key Concepts

• API Gateway: A centralized management tool that sits between a client and a collection of backend services to handle routing, security, and traffic management.
• Usage-Based Billing: A pricing model where customers are charged based on their actual consumption of a service (e.g., API calls, tokens, or data volume).
• Open Meter: An open-source infrastructure project designed for metering and billing, recently acquired by Kong.
• Kong Connect: The cloud-based, managed version of Kong’s API gateway platform.
• Token-Based Metering: A specific method of tracking usage in AI applications, where costs are calculated based on input and output tokens processed by LLMs (Large Language Models).
• Entitlements: The specific features or usage limits granted to a customer based on their subscription plan.

1. Main Topics and Key Points

The video focuses on the integration of Open Meter into the Kong ecosystem to solve the challenge of monetizing APIs and AI services.

• The Problem: Developers often struggle to track usage accurately at scale, leading to revenue loss or complex, manual billing processes.
• The Solution: By using an API gateway (Kong) as a proxy, developers can intercept traffic, track usage via Open Meter, and generate precise invoices.
• AI Context: The speakers emphasize that "there is no AI without API." As AI agents become more prevalent, tracking token consumption (which is often asymmetric, with output tokens costing more than input tokens) is critical for sustainable business models.

2. Real-World Applications

• AI Model Monetization: Companies using multiple LLMs (e.g., GPT, Claude, Gemini) can use Kong to route traffic and Open Meter to track token usage across different models, allowing for tiered pricing.
• SaaS Billing: Startups can move away from "database and a prayer" billing methods to automated, infrastructure-level metering that integrates with payment processors like Stripe.

3. Step-by-Step Methodology (Implementation)

1. Gateway Setup: Create a gateway in Kong Connect (serverless or self-managed) and configure it as a proxy for the target API.
2. Authentication: Enable a plugin (e.g., Key Authentication) to identify specific consumers/customers.
3. Metering Configuration:
   • Define Features (e.g., "Tokens").
   • Create Plans (e.g., "Premium Plan") with specific Rate Cards (e.g., $1 per unit).
4. Subscription Mapping: Map a specific consumer to a plan.
5. Tracking & Invoicing: As the consumer makes requests, the gateway tracks the usage, updates the meter, and generates an invoice based on the configured rate.

4. Key Arguments

• Infrastructure-Level Tracking: The speakers argue that metering should happen at the infrastructure layer (the gateway) rather than the application code layer. This prevents "ticking time bombs" where application logic becomes bloated with billing code.
• Precision: Using an infrastructure-based approach ensures that billing is accurate, reducing disputes and providing clear visibility into consumption patterns.

5. Notable Quotes

• "There is no AI without API." — Emphasizing the necessity of API gateways in the modern AI development stack.
• "Think of it like electricity... you start using your electricity and then you start metering it." — Explaining the concept of usage-based billing.

6. Logical Connections

The workflow connects the API Gateway (the traffic controller) with Open Meter (the measurement engine). By placing the metering service at the gateway level, the system gains a "single source of truth" for usage data, which is then fed into a billing cycle, ultimately resulting in a financial transaction.

7. Synthesis and Conclusion

The integration of Open Meter into Kong provides a robust, scalable, and open-source solution for developers to monetize their services. By shifting billing from a manual, application-level task to an automated, infrastructure-level process, developers can focus on building AI agents and APIs while ensuring they are fairly compensated for their resource consumption. The project encourages community contribution, offering opportunities for developers to document their use cases and contribute to the open-source codebase.