New #1 open-source AI, new deepfake tools, image editor beats GPT-4o, free deep researcher

Key Concepts

• Video Segmentation: Isolating specific objects within a video.
• Semantic Image Editing: Modifying images using natural language prompts.
• Deep Fake Lip Sync: Generating realistic videos of people speaking audio they didn't originally say.
• Hybrid Reasoning Models: AI models that can switch between "thinking" and "non-thinking" modes for problem-solving.
• Deep Research AI: AI agents that can autonomously search the internet, read web pages, and write research reports.
• AI Music Generation: Creating music using artificial intelligence.
• Video Clothes Swapping: Replacing clothing in a video with a different garment.

Edgetam: Efficient Video Segmentation on Mobile Devices

• Main Topic: Edgetam, an AI model for video segmentation designed for efficiency on mobile devices.
• Key Points:
  • Tracks objects in videos by selecting them in the first frame.
  • Generates a mask or outline of the object and tracks it throughout the video.
  • Runs 22 times faster than SAM 2, achieving 16 frames per second on an iPhone 15 Pro Max without quantization.
  • While not the most accurate segmentation tool, it's the only one that runs effectively on a mobile device.
• Technical Details: Based on SAM 2 but optimized for efficiency.
• Real-World Application: Video editing, augmented reality applications on mobile devices.
• Availability: GitHub repository with instructions for local download and use.

ICEdit: In-Context Image Editing with Natural Language

• Main Topic: ICEdit, an AI model that edits images using natural language prompts.
• Key Points:
  • Edits images based on text prompts, such as adding objects, changing backgrounds, or altering styles.
  • Demonstrates superior performance compared to Gemini and GPT40 in certain image editing tasks.
  • Maintains consistency in facial features and poses during edits.
• Examples:
  • Adding a cup of tea and closing eyes in a portrait.
  • Changing a background to Hawaii and altering clothing.
  • Removing watermarks and objects from photos.
  • Turning photos into watercolor paintings or pencil sketches.
• Comparison: Outperforms Gemini and GPT40 in specific tasks like adding accessories and maintaining facial consistency.
• Availability: Free Hugging Face space for online use and a GitHub repository for local installation.

Hydream E1: Open-Source Image Editing

• Main Topic: Hydream E1, an open-source AI model for image editing using natural language.
• Key Points:
  • Builds on the Hydream image model.
  • Edits images based on text prompts, such as changing hair color or converting to a specific style.
  • Demonstrates competitive performance compared to other AI image editors.
• Examples:
  • Changing hair color to red.
  • Converting an image to Ghibli style.
  • Replacing an apple with an orange.
  • Adding autumn leaves to an image.
• Availability: GitHub repository with instructions for local download and use, and the model is available on Hugging Face.

Fantasy Talking: Realistic Deep Fake Lip Sync

• Main Topic: Fantasy Talking, an AI model that generates realistic videos of people speaking from a single image and audio clip.
• Key Points:
  • Animates the entire photo, including head and body movements, to match the audio.
  • Creates more natural-looking videos compared to previous lip-sync tools.
  • Works with realistic photos, 3D characters, and animals.
  • Based on Alibaba's Wand 2.1 video generator.
• Comparison: While not as accurate in lip sync as Omnihuman by Byte Dance, it's free and open source.
• Availability: GitHub repository with instructions for local download and use, integrated into Comfy UI.

Gamma: AI-Powered Content Creation Platform

• Main Topic: Gamma, an AI-powered platform for creating presentations, websites, and social media content.
• Key Points:
  • Uses AI to generate content from simple prompts.
  • Allows users to create presentations, websites, social media carousels, and documents.
  • Offers easy export to Google Slides or PowerPoint.
  • Has over 50 million users.
• Real-World Application: Content creation for marketers, agencies, educators, and consultants.
• Availability: Free trial available at gamma.app.

Quen 3: Open-Source Hybrid Reasoning Model by Alibaba

• Main Topic: Quen 3, a family of open-source hybrid reasoning models released by Alibaba.
• Key Points:
  • Matches or surpasses leading models from OpenAI, Google, and Deepseek in math, coding, and reasoning tasks.
  • Supports two modes: "thinking" (step-by-step reasoning) and "non-thinking" (instant answer).
  • Supports 119 languages and dialects.
  • Offers a range of models from 0.6 billion to 235 billion parameters.
  • Cost-effective compared to other leading AI models.
• Technical Details: Hybrid approach to problem-solving, supports both reasoning and non-reasoning modes.
• Benchmarks: Performs well on Live Codebench, Codeforces, and BFCL benchmarks.
• Availability: Models available on Hugging Face and Model Scope, GitHub repository with instructions for local use, and online platform called Quen Chat.

Microsoft Phi-3: Small Reasoning Models

• Main Topic: Phi-3, a family of small reasoning models released by Microsoft.
• Key Points:
  • Good at solving complex problems using step-by-step thinking.
  • Three models: Phi-3, Phi-3 Reasoning, and Phi-3 Reasoning Plus.
  • Each model has 14 billion parameters.
  • Can potentially run on laptops or mobile devices.
• Benchmarks: Performs close to GPT40 and 03 Mini on certain benchmarks.
• Availability: Models available for download on Azure AI and Hugging Face, with free Hugging Face spaces for use.

Web Thinker: Open-Source Deep Research AI

• Main Topic: Web Thinker, a free and open-source AI that can search the internet, read web pages, and write research reports.
• Key Points:
  • Autonomously searches the web, gathers information, and writes research reports.
  • Aggregates, cross-references, and verifies information.
  • Powered by Alibaba's QWQ model.
  • Outperforms base models in specialized knowledge domains.
• Benchmarks: Performs well on the Humanity's Last Exam benchmark and in scientific report generation.
• Availability: GitHub repository with instructions for local download and installation, models available for download.

Suno 4.5: Advanced AI Music Generator

• Main Topic: Suno 4.5, the latest version of the Suno AI music generator.
• Key Points:
  • Improved vocals with greater depth, emotion, and dynamic range.
  • Generates more complex sounds with natural tone shifts and instrumental layering.
  • Better prompt understanding.
• Availability: Requires a paid subscription starting at $8 per month.

Alibaba's Video Clothes Swapper

• Main Topic: An AI tool by Alibaba that replaces clothes in a video with a new garment.
• Key Points:
  • Replaces clothing in videos accurately, maintaining details and light reflections.
  • Can swap entire outfits or individual pieces of clothing.
  • Demonstrates superior performance compared to other AI clothes swapping tools.
• Availability: Code is coming soon.

Synthesis/Conclusion

This week in AI saw significant advancements across various domains, including video segmentation, image editing, deep fake lip sync, reasoning models, deep research, music generation, and video clothes swapping. Notably, Alibaba released several powerful open-source tools, including Quen 3 and Web Thinker, challenging the dominance of closed-source models from companies like OpenAI and Google. The focus on efficiency and accessibility is evident in models like Edgetam and Phi-3, which can run on consumer devices. These developments highlight the rapid pace of innovation in AI and the increasing availability of sophisticated tools for a wide range of applications.