The best FREE AI text to speech & voice cloner is here!

Key Concepts:

• Vibe Voice: A free and open-source voice cloning and text-to-speech generator by Microsoft.
• Voice Cloning: Replicating a person's voice using a short audio sample.
• Context-Aware Expression: Automatically applying appropriate emotions and expressions to the generated voice based on the transcript.
• ComfyUI: A popular platform for running open-source image, video, and audio generators.
• Custom Nodes: Extensions for ComfyUI that add new functionalities.
• Text-to-Speech (TTS): Converting written text into spoken audio.
• Hugging Face: A platform for sharing and using machine learning models.
• VRAM: Video Random Access Memory, used by GPUs for processing.
• Parameters: Variables that a machine learning model learns during training.
• Diffusion Steps: The number of iterations an AI model performs to generate audio.
• Seed: A starting point for the AI generation process.
• CFG Scale: Controls how closely the AI follows the prompt.
• Temperature: Determines the randomness and variability of the output.
• Top P: Similar to temperature, influences the diversity of the output.

1. Introduction to Vibe Voice

• Vibe Voice is a free and open-source text-to-speech generator developed by Microsoft.
• It excels at voice cloning, generating outputs over 90 minutes long, and supporting up to four distinct speakers.
• The video provides a step-by-step guide on how to install and use Vibe Voice on your computer.

2. Voice Cloning Capabilities

• Accurate Voice Cloning: The tool can accurately clone voices using short audio clips (e.g., 22 seconds of Trump's voice, 9 seconds of Sam Altman's voice).
  • Example: Cloned Trump and Sam Altman's voices to create a conversation about AI.
• Context-Aware Expression: Vibe Voice can apply appropriate emotions and expressions based on the context of the transcript.
  • Example: A monologue with happiness, sadness, and anger, where the cloned voice accurately reflected the changing emotions.
  • Example: A conversation between two people arguing, where the cloned voices conveyed the appropriate tone and emotions.
• Multiple Languages and Accents: Vibe Voice supports multiple languages and can capture accents.
  • Example: Generating speech in Japanese, Spanish, and German.
  • Example: Mixing English, Chinese, and Korean in a single sentence with a Japanese accent.
  • Example: Cloning voices with Aussie and Indian accents.

3. Advanced Features and Demos

• Music Integration: Vibe Voice can capture and generate background music from the input audio.
  • Example: Generating a podcast intro with background music.
• Long Audio Generation: The tool can generate audio outputs exceeding 90 minutes.
  • Example: Generating a 93-minute-long podcast episode.
• Benchmark Performance: Vibe Voice is preferred over other text-to-speech models like Gemini 2.5 Pro, 11 Labs version 3, Higs audio, and Sesame, according to a human preference benchmark.

4. Model Specifications

• Three models are available:
  • 0.5 billion parameter model (for real-time streaming, not yet released).
  • 1.5 billion parameter model (released).
  • 7 billion parameter model (released).
• The 1.5B model has a larger context length and can generate audio up to 90 minutes long.
• The 7B model produces higher quality audio but has a shorter context length.
• Both models can be run on consumer-grade GPUs.

5. Usage and Installation

• Hugging Face Demo: A free online demo is available on Hugging Face, but it does not allow users to input their own voices for cloning.
• Local Installation with ComfyUI: The recommended method is to install Vibe Voice locally using ComfyUI for unlimited use and custom voice cloning.
• ComfyUI Installation Steps:
  1. Navigate to the custom_nodes folder in your ComfyUI installation.
  2. Open a command prompt in the custom_nodes folder.
  3. Run git clone [repository URL] to clone the Vibe Voice ComfyUI custom node repository.
  4. Restart ComfyUI.
• ComfyUI Workflow:
  • Load audio clips for each speaker.
  • Enter the transcript with speaker designations (e.g., [1] Hello).
  • Select the desired model (1.5B or 7B).
  • Adjust settings like attention type, diffusion steps, seed, CFG scale, temperature, and top P.
  • Run the workflow to generate the audio.

6. ComfyUI Settings Explained

• Free Memory After Generate: Determines whether the model is unloaded from GPU memory after generation.
• Diffusion Steps: The number of steps the AI model takes to generate audio (20 is a good starting point).
• Seed: The starting point for the generation; using a fixed seed ensures the same output.
• CFG Scale: Controls how literally the AI follows the prompt.
• Temperature: Determines the randomness and variability of the output.
• Top P: Similar to temperature, influences the diversity of the output.

7. Alternative Text-to-Speech Generators

• F5TS and Zonos are other open-source text-to-speech generators that allow control over expressions and emotions.

8. Art List Integration

• Art List is a platform for video creators offering digital assets and AI tools.
• It includes features like video generation, AI voiceovers, image generation, and a voice-over tool with customizable settings and unique voice effects.

9. Conclusion

• Vibe Voice is a powerful and versatile text-to-speech and voice cloning tool.
• It offers accurate voice cloning, context-aware expression, multiple language support, and music integration.
• The tool can be used online via Hugging Face or installed locally using ComfyUI for unlimited use.
• The video provides a comprehensive guide on how to install and use Vibe Voice, along with explanations of key settings and features.