Insane 3D model generator, emotional TTS, AI eraser, 3D upscaler, Qwen3 beats all, 4D videos

Key Concepts

• 3D Model Generation/Upscaling: Creating or enhancing 3D models from images or existing models.
• Video Object Tracking/Segmentation: Identifying and tracking specific objects within video footage.
• Object Removal: Erasing objects from images or videos, including shadows and reflections.
• Humanoid Robotics: Development of robots resembling humans in form and function.
• Interactive World Generation: Creating dynamic, explorable 3D environments from images.
• Large Language Models (LLMs): AI models with a vast number of parameters, used for text generation, coding, and reasoning.
• Mixture of Experts (MoE): An LLM architecture where multiple sub-models (experts) work together.
• Context Length: The amount of text or code an LLM can process at once.
• Hierarchical Reasoning: An AI architecture that mimics the human brain's layered thinking process.
• Text-to-Speech (TTS): Converting text into spoken audio, including voice cloning and emotion.
• 4D Video Generation: Creating 3D video that can be viewed from different angles.
• Slide Design Enhancement: Improving the visual appeal and clarity of presentation slides.

Microsoft David: Accurate 3D Human Prediction

• Main Topic: Microsoft's David AI model for predicting 3D information from 2D images of humans.
• Key Points:
  • David predicts depth information, surface normals (orientation), and segmentation (masking).
  • Highly accurate segmentation, even with fine details like hair strands.
  • Applicable to both images and videos.
  • Captures wrinkles and facial details in depth and normal estimations.
• Technical Details:
  • Uses a dense prediction transformer architecture.
  • Employs three lightweight convolutional heads for segmentation, depth, and normal estimation.
• Resources:
  • Dataset and code are publicly available for download and local execution.

SEC: Superior Video Object Tracking

• Main Topic: SEC, a new AI for tracking and separating objects in videos using text prompts.
• Key Points:
  • Tracks specified objects accurately, even in high-action scenes with obstructions.
  • Outperforms previous object segmentation tools like SAM 2 in consistency and accuracy.
  • Handles scene cuts and complex scenarios effectively.
• Examples:
  • Tracking a white soccer ball in a fast-paced game.
  • Segmenting a specific violin held by a performer in a crowded scene.
  • Tracking a small gray rabbit among many rabbits.
• Comparison with SAM 2: SEC maintains tracking consistency in high-action scenes where SAM 2 fails.
• Benchmarks: SEC outperforms other video tracking tools like Samurai and SAM 2 across various benchmarks.
• Resources: Dataset available on Hugging Face; code and instructions on GitHub for local use.

Object Clear: Intelligent Object Removal

• Main Topic: Object Clear AI for removing objects from images and videos, including shadows and reflections.
• Key Points:
  • Removes objects and their associated effects (shadows, reflections) automatically.
  • Extrapolates and fills in the area behind the removed object seamlessly.
  • Two methods for specifying objects: brush strokes or clicking.
• Examples:
  • Removing a cyclist and their shadow from a photo.
  • Erasing a corgi and its shadow.
  • Removing a car and filling in the background.
  • Removing a ship and its reflection.
• Hugging Face Space: Free online demo available for testing.
• Code Availability: Code and instructions on GitHub for local installation and use.

Unitree R1: Affordable Humanoid Robot

• Main Topic: Unitree R1, a low-cost, acrobatic humanoid robot.
• Key Points:
  • Starts at $5,900 and weighs 25 kg (55 lbs).
  • Performs acrobatic feats like cartwheels and handstands.
  • Integrated with a large multimodal model for voice interaction and vision capabilities.
• Capabilities:
  • Can punch, kick, and run.
  • Customizable outer appearance.

Um: Interactive World Generator

• Main Topic: Um, an AI that generates realistic and dynamic interactive worlds from images.
• Key Points:
  • Uses an input image as a reference to construct a 3D world.
  • Utilizes Alibaba's Juan 1 2.1 for video generation.
  • Responds to W, A, S, D, and arrow keys for navigation.
• Comparison with Other Generators: Similar to Hunyan GameCraft and Mirage, but Um has released its code.
• Resource Requirements: Trained with an A100 (80GB VRAM), but a quantized version is planned for consumer GPUs.
• Code Availability: GitHub repo with instructions for local download and execution.

Chat LLM by Abacus AI

• Main Topic: Chat LLM, an all-in-one platform for accessing various AI models, image generators, and video generators.
• Key Features:
  • Seamlessly switch between different AI models.
  • Artifacts feature for side-by-side preview of generations.
  • Deep Agent feature for complex autonomous tasks (PowerPoints, websites, research reports).
• Pricing: $10 per month for access to all features.

Alibaba Quen 3: Top Open-Source LLM

• Main Topic: Alibaba's Quen 3, a high-performing open-source LLM.
• Key Points:
  • Outperforms other non-reasoning models, including proprietary models like Claude and GPT.
  • Excels in logical reasoning, text comprehension, mathematics, science, coding, and tool usage.
• Technical Details:
  • 235 billion parameters.
  • Mixture of Experts (MoE) architecture.
  • 262,000 token context length.
• Benchmarks: Significantly outperforms other models in competitive math and Arena Hard benchmarks.
• Arc AGI Performance: Achieves a score of 41.8% on Arc AGI, demonstrating strong learning and pattern recognition abilities.
• Accessibility: Free and open-source; available for online use and local download.
• Use Cases: Summarizing articles, writing emails, research, travel planning.
• API Pricing: $1.2 per 1 million tokens.

Alibaba Quen 3 Coder: Open-Source Coding LLM

• Main Topic: Alibaba's Quen 3 Coder, an open-source LLM specialized for coding.
• Key Points:
  • Outperforms closed-source coding models from OpenAI and Anthropic.
• Technical Details:
  • 480 billion parameter Mixture of Experts model.
  • 250k token context length, expandable to 1 million with extrapolation.
• Benchmarks: Outperforms leading models in coding and agentic tool use benchmarks.
• Examples:
  • Creating a LinkedIn clone in a single HTML file.
  • Simulating probability experiments with interactive visualizations.
  • Generating a futuristic city with interactive controls using 3JS.
  • Creating an interactive high school physics course with simulations.
• Accessibility: Free and open-source; available for online use and local download.

Hierarchical Reasoning Model (HRM): Efficient AI Reasoning

• Main Topic: Hierarchical Reasoning Model (HRM), a new AI architecture for efficient reasoning.
• Key Points:
  • Mimics the human brain's layered thinking process with multi-time scale processing.
  • Uses a recurrent architecture for complex calculations without instability.
  • Solves complex problems in one quick pass, unlike chain-of-thought models.
• Architecture:
  • High-level module for abstract thinking.
  • Low-level module for detailed work.
• Performance:
  • Outperforms larger models (Gemini, Claude, Grok) on complex puzzles (Sudoku, Maze Hard, Arc AGI) with only 27 million parameters.
  • Requires less training data (around 1,000 samples).
• Implications: Suggests that better architecture design is more important than model size and data volume for achieving AGI.
• Code Availability: Code released on GitHub for running and testing.

Higs Audio V2: Open-Source Text-to-Speech

• Main Topic: Higs Audio V2, an open-source text-to-speech generator with voice cloning and emotion capabilities.
• Key Points:
  • Supports multiple speakers, long audio generation, voice cloning, and emotion.
  • Trained on over 10 million hours of audio.
• Capabilities:
  • Detects speakers, clones their voices, and generates speech in different languages.
  • Conveys emotions effectively.
• Benchmarks: Outperforms other TTS models (11 Labs, Sesame 1B) in handling emotions.
• Examples:
  • Cloning voices and generating conversations between Shrek, Donkey, and Fiona.
  • Cloning a voice and generating speech with different emotions (sad and excited).
• Accessibility: Free online demo available on Hugging Face; code on GitHub for local installation.

DiffuMan 4D: 4D Video Generation

• Main Topic: DiffuMan 4D, an AI for generating 4D videos of people from sparse view videos.
• Key Points:
  • Creates coherent 3D renders from a few videos taken at different angles.
  • Estimates and fills in areas not captured by the original videos.
• Performance: Generates higher quality videos compared to competitors like Longvall Cap.
• Architecture: Uses a spatial temporal diffusion model to generate the video.
• Resource Requirements: Requires fewer input videos than other methods (four videos vs. 48 for Longvall Cap).
• Code Availability: Code and dataset planned for release on GitHub.

Design Lab: Slide Design Enhancement

• Main Topic: Design Lab, an AI that enhances the visual appeal of presentation slides.
• Key Points:
  • Reorganizes slide elements and improves overall design.
  • Clears up backgrounds and improves text legibility.
  • Adds color and vibrance to slides.
• Architecture:
  • Design reviewer identifies and fixes design issues.
  • Design contributor suggests improvements.
• Availability: "Coming soon" according to the website.

Ultra 3D: High-Quality 3D Model Generator

• Main Topic: Ultra 3D, a top-tier AI for generating detailed 3D models from images.
• Key Points:
  • Converts 2D images into highly detailed 3D models.
  • Outperforms competitors like Trellis in detail and accuracy.
• Examples:
  • Generating detailed 3D models of warriors, characters, and objects.
• Accessibility: Currently available through the HTM 3D platform (online platform with credits).

Elevate 3D: 3D Model Upscaler

• Main Topic: Elevate 3D, an AI for enhancing the quality of 3D models.
• Key Points:
  • Refines texture and geometry of 3D models.
  • Adds details and sharpens blurry models.
• Technical Details: Refines both texture and geometry separately.
• Accessibility: Code and instructions available on GitHub for local use.

Conclusion

This week in AI saw significant advancements across various domains, including 3D model generation and upscaling, video object tracking, object removal, humanoid robotics, interactive world generation, and large language models. Notably, Alibaba's open-source Quen 3 models have emerged as strong contenders, rivaling proprietary models in both general language understanding and coding. The Hierarchical Reasoning Model (HRM) demonstrates that efficient AI reasoning can be achieved with smaller models and less data through innovative architectural design. These developments highlight the rapid pace of innovation in AI and the increasing accessibility of powerful tools through open-source initiatives.