AI deepfakes just got even more real

AI Avatar Generation Tools: HeyGen Avatar 4 vs. Mirage - A Detailed Review

Key Concepts:

• AI Avatar Generation
• Lip-syncing
• Motion Gesture Control
• Text-to-Speech (TTS)
• Realistic Avatar Animation
• Expressive Video Generation
• Hard Cuts/Video Stitching
• Uncanny Valley

HeyGen Avatar 4 Review

Main Topics and Key Points:

• Overview: HeyGen Avatar 4 is a new feature that allows users to upload an image of a character and audio, then animate the character to lip-sync with the audio. It also includes motion control for gestures.
• Free Trial: Offers a free trial with a limited number of videos (3 in this case).
• Avatar 4 Feature: Allows uploading custom character images and audio.
• Text-to-Speech: Offers various TTS voices, but they sound robotic.
• Custom Audio Upload: Supports uploading custom audio for more realistic voice.
• Motion Prompts: Users can add prompts to describe character motion, especially hand gestures and facial expressions.
• Generation Time: Video generation can take a significant amount of time (e.g., 10 minutes for a 6-second video).
• Limitations:
  • Flawed Generations: Errors occur, such as objects appearing unexpectedly.
  • Rigid Animations: Animations often appear too rigid and robotic.
  • Poor Gesture Control: Custom prompts for gestures are often ignored or poorly executed.
  • Inability to Handle Complex Emotions: Struggles with expressions like laughter or sadness.
  • Singing Inaccuracies: Fails to accurately animate singing.
• Language Support: Lip-syncing works reasonably well in different languages.
• Comparison to Omnihuman: Omnihuman is presented as a more realistic and natural alternative.

Important Examples:

• Dude with Serious Look: The first example, where the character was supposed to talk with a serious look and give a thumbs up, had errors with objects appearing and disappearing.
• Woman Singing: The singing example was deemed "pretty bad," with mediocre lip-sync and a lack of passionate expression.
• Sad Dude in Dark Room: The attempt to generate a sad character failed to produce a convincing expression.

Key Arguments:

• HeyGen Avatar 4's motion gesture control is limited and doesn't consistently follow prompts.
• The generated avatars often look too rigid and robotic compared to alternatives like Omnihuman.
• HeyGen Avatar 4 struggles with complex emotions and singing.

Notable Quotes:

• "Hey there here's how I sound Hope you like it" - Example of HeyGen's text-to-speech voices.
• "Well that was pretty bad I must say She's definitely not singing passionately Even the lips sync is just mediocre" - Critique of the singing example.

Technical Terms:

• Lip-sync: Matching the movement of a character's lips to the audio.
• Motion Gesture Control: The ability to control the movements and gestures of an avatar.

Mirage Studio Review

Main Topics and Key Points:

• Overview: Mirage Studio claims to generate expressive videos of characters, including laughter and singing.
• Free Credits: Offers free credits for initial use (600 credits, enough for 3-4 videos).
• Audio Input: Supports uploading audio clips or generating audio using TTS.
• Segmented Video Generation: A major limitation is that videos are broken down into segments of a maximum of 4 seconds long and then stitched together.
• Slow Generation Process: The process of generating videos, including the "audition" phase, is extremely slow.
• Limitations:
  • Hard Cuts: The stitching of 4-second clips results in noticeable and jarring hard cuts.
  • Inconsistent Laughter Generation: The quality of laughter generation varies and is not always as good as the demos.
  • Failure to Animate Singing: Fails to accurately animate characters singing, despite showcasing this capability in demos.
• Text-Prompted Actor Generation: Can generate an actor from a text prompt, but the results may not always match the description.
• Language Support: Lip-syncing works well for regular speech in different languages.

Important Examples:

• Laughter Clip: The initial laughter clip example was more natural than HeyGen, but the hard cut was a significant issue.
• Rap Song: While the movements were more natural than HeyGen, the hard cuts were still a problem.
• Singing Example: The character did not sing the lyrics of the song, despite the tool detecting some lyrics.
• Sad Young Man: The generated character conveyed some sadness, but the result was still not great and had uncanny elements.

Key Arguments:

• The 4-second clip limitation and resulting hard cuts are a major drawback of Mirage Studio.
• Mirage Studio's ability to generate realistic laughter and singing is inconsistent and often falls short of its demos.
• While lip-syncing for regular speech is good, there are many faster alternatives.

Notable Quotes:

• "This is my necklace It's mine Stop Stop saying it's a song" - Example audio clip used to test laughter generation.
• "I shouldn't Oh my goodness It's too much I can't breathe" - Another audio clip used to test laughter generation.

Technical Terms:

• Hard Cuts: Abrupt transitions between video clips.
• Uncanny Valley: The feeling of unease or revulsion that can arise when encountering a human-like representation that is not quite realistic.

Comparison and Conclusion

Logical Connections:

• The review transitions from HeyGen Avatar 4 to Mirage Studio because HeyGen's inability to handle laughter and sad emotions is a segue to Mirage's claims of generating expressive videos.
• Both tools are compared to Omnihuman as a benchmark for realistic avatar animation.

Synthesis/Conclusion:

Both HeyGen Avatar 4 and Mirage Studio, despite the hype, have significant limitations. HeyGen Avatar 4 produces rigid and robotic animations with limited gesture control, while Mirage Studio suffers from the 4-second clip limitation and inconsistent results. Neither tool fully lives up to its advertised capabilities, and existing alternatives like Omnihuman may offer more realistic results. The reviewer emphasizes the importance of critical evaluation of AI tools, even those heavily promoted by influencers.