Build and Deploy a Full Stack ElevenLabs Clone with Next.js 16

Key Concepts

• Full-Stack AI Voice Generation: Building a SaaS platform ("Resonance") for text-to-speech (TTS) and voice cloning.
• Self-Hosted AI: Using the "Chatterbox" model on serverless GPUs (Model.com) instead of paid third-party APIs.
• Multi-Tenancy: Using Clerk for authentication and organization-based isolation.
• Usage-Based Billing: Integrating Polar for metered billing (per character/voice creation).
• Infrastructure: Next.js 16, Tailwind CSS v4, Prisma (PostgreSQL), Cloudflare R2 (S3-compatible storage), and Railway for deployment.
• Type Safety: TRPC for end-to-end type-safe APIs and Zod for schema validation.
• Error Monitoring: Sentry for real-time error tracking and logging.

1. Project Architecture & Foundation

• Framework: Next.js 16 with Tailwind CSS v4 and Shadcn UI.
• Authentication: Clerk handles multi-tenant workspaces. The proxy.ts (formerly middleware.ts) ensures users are authenticated and assigned to an organization before accessing the app.
• Database: Prisma ORM with PostgreSQL. A singleton pattern is used for the PrismaClient to prevent connection pool exhaustion during hot reloads.
• Environment Management: T3-OSS env-nextjs is used to enforce type-safe environment variables, causing the app to fail at build time if variables are missing.

2. Voice Generation Pipeline

• Model Hosting: The Chatterbox TTS model is hosted on Model.com. It uses a FastAPI Python server to expose endpoints for generation.
• API Integration: The app uses open-api-fetch to generate TypeScript types from the model's OpenAPI spec, ensuring the Next.js frontend communicates with the Python backend in a type-safe manner.
• Storage: Audio files are stored in Cloudflare R2. The app uses signed URLs to stream audio securely, ensuring files are not publicly accessible.
• Generation Flow:
  1. User inputs text and selects a voice.
  2. TRPC mutation triggers the Chatterbox API.
  3. Audio is generated, uploaded to R2, and the metadata is saved in PostgreSQL.
  4. The frontend uses WaveSurfer.js to visualize the waveform and provide playback controls.

3. Monetization & Billing (Polar)

• Metered Billing: Polar is used to track usage. Two meters were created:
  • text_to_speech_characters: Sum aggregation based on character count.
  • voice_creation: Count aggregation for custom voice clones.
• Subscription Gating: TRPC procedures (e.g., createGeneration) check for active subscriptions via Polar’s API. If a user is not subscribed, the app throws a FORBIDDEN error, triggering a toast notification that redirects the user to the Polar checkout portal.

4. Step-by-Step Implementation Highlights

• Voice Selection: A custom VoiceSelector component handles both system voices (seeded via script) and custom team voices. It uses dicebear for unique avatar generation.
• Recording: The VoiceRecorder uses RecordRTC and WaveSurfer to provide real-time visual feedback during audio capture.
• CI/CD: The project is deployed on Railway. Every pull request triggers a preview environment, and Code Rabbit provides AI-driven code reviews.
• Error Monitoring: Sentry is integrated via a wizard, providing stack traces, session replays, and structured logging for production debugging.

5. Notable Quotes & Perspectives

• "By using Clerk, it's almost like you hired a professional security team for you."
• "Server components to me are very good if they're used for what they're made for... I just think of them as API routes which are able to return JSX."
• "The more type-safe you are, the better results AI will produce in your codebase."

6. Synthesis & Conclusion

Resonance is a production-ready SaaS architecture that demonstrates how to build a complex AI application without relying on expensive third-party APIs. By combining TRPC for type safety, Polar for effortless metered billing, and Model.com for self-hosting open-source models, the project achieves a scalable, cost-effective, and highly maintainable structure. The use of Sentry and Railway ensures that the application is observable and easily deployable, making it a robust template for modern AI-driven SaaS products.