Gemini 3.0 (X58 New Checkpoint TESTED): Google launched a new Checkpoint for Gemini 3 & I TESTED IT!

Key Concepts

• Gemini Pro Checkpoint (X28, 2 HT): Experimental versions of Google's Gemini Pro AI model, with X28 being a newer, more advanced iteration.
• AI Studio: Google's platform where these AI models are tested and accessed.
• Gemini 3.0 Pro: The anticipated next major version of the Gemini Pro model.
• Surf Shark VPN: A Virtual Private Network service offering secure and unrestricted internet access.
• Benchmarks: Standardized tests used to evaluate the performance of AI models across various tasks (e.g., floor plan generation, SVG creation, 3D model generation, code generation).
• Consistency/Low Variance: The ability of an AI model to produce similar, high-quality outputs for the same prompt, reducing variability.
• Sonnet: Another AI model, often used as a point of comparison for performance and pricing.
• Degree of Separation Simulation: A complex task involving generating a network graph with nodes and connections, often used to illustrate social network concepts.
• UI Designing: The capability of an AI model to generate user interface components or entire UI designs.
• Vibe Coded: A term used to describe AI-generated content that appears generic, uninspired, or lacking thoughtful design, often characterized by specific aesthetic choices (e.g., purple hues).
• Tool Calling: The ability of an AI model to interact with external tools or APIs to perform specific actions or retrieve information.
• Human Relay Mode (from Rue): A specific mode or framework for tool calling, likely involving human oversight or interaction.
• Gemini CLI: Command Line Interface for interacting with Gemini models, anticipated to include tool calling capabilities.

---

Introduction to the New Checkpoint The video discusses a new Gemini Pro checkpoint, designated X28, which appeared in AI Studio shortly after Google halted access to the previously tested 2 HT checkpoint. The X28 checkpoint is speculated to be the precursor to the upcoming Gemini 3.0 Pro and is reportedly even more capable than the 2 HT version. The presenter conducted a series of 11 benchmark questions and additional tests to evaluate its performance.

---

Sponsor Spotlight: Surf Shark VPN The video is sponsored by Surf Shark VPN, highlighted as a solution for accessing AI models restricted by region and ensuring online security.

• Access & Servers: Offers access to over 3,200 servers in more than 100 countries, facilitating content access and API discovery for projects.
• Security & Privacy: Protects data from leaks, shields personal information, and safeguards devices from malware. It supports use across all devices (desktop, mobile, tablet) with a single account.
• Ad-Free Browsing: Includes a private search engine that allows browsing without ads or a digital footprint.
• No-Logs Policy: Surf Shark passed an independent no-logs audit in June, assuring users that their activity is never tracked.
• Special Offer: Viewers can get four extra months using the code C code king at checkout, backed by a 30-day money-back guarantee.

---

Performance Benchmarks and Specific Generations

The X28 checkpoint was tested across various generation tasks, demonstrating significant improvements.

• Floor Plan Generation:

  • Cohesiveness and Realism: The generated floor plan was described as "much more cohesive" and making "much more sense" compared to previous versions. Doors were correctly placed within walls, and the overall layout was logical.
  • Furniture Design: The furniture design was "really very good."
  • Interactive Features: It included lighting controls, allowing users to visualize the plan at different times with accurate shadow rendering.
  • Consistency: The model showed "very good" consistency, producing "almost a similar thing" each time for the same prompt, with "a lot lower" variance compared to models like Sonnet.
  • Interactivity: A notable new feature was the ability to drag around the furniture, indicating advanced interactive generation capabilities. The presenter estimated it to be "50% better than the previous one."

• SVG Panda:

  • Improved Realism: The SVG generation of a panda was "amazingly good," more cohesive, and adhered better to the prompt. The panda realistically appeared to be eating a burger rather than just holding it, and the burger itself looked "really good."

• 3.js Pokeball:

  • Enhanced Visuals: The model "nailed it again" in generating a Pokeball in 3.js. While previously good, it was now "much better," featuring colors in the background that looked "more cohesive."

• Minecraft Game:

  • Detailed Environment: The generated Minecraft game environment was "very similar" to previous attempts but now included rivers and better lighting, making it "pretty great to see" and "really very good" for a one-shot generation.

• Majestic Butterfly:

  • High-Quality Scene: The generation of a majestic butterfly flying in a garden was "so good" and "amazing," showcasing detailed rocks, flowers, and a great environment. A minor issue noted was that the butterfly "clips sometimes."

• CLI Tool in Rust & Blender Script:

  • Functional Code Generation: The generation of a CLI tool in Rust "worked well."
  • Consistent Scripting: The Blender script for the Pokeball was "very similar to the previous generation," indicating consistent performance in script generation.

---

Overall Performance Assessment The presenter estimates the X28 checkpoint to have a 5% or 10% improvement over the last checkpoint tested. This is considered a "real leap" from currently available models like Sonnet, with the presenter likening it to a "Sonnet moment" for its significant advancement.

---

Advanced Capabilities and Additional Tests

Beyond the standard benchmarks, the X28 checkpoint demonstrated impressive capabilities in more complex tasks.

• Degree of Separation Simulation:

  • Accuracy and Aesthetics: The model "really nailed it on the first try" in generating a simulation for degree of separation with nodes and a friendship slider. It was described as "so aesthetic" and working "pretty accurately."
  • Interactive Features: Users could regenerate the network and check with random selections.
  • Superior UI Design: Unlike many other models that produce "ugly purple hue" and "vibe coded" outputs, this generation looked "pretty good," used "different fonts," and showed thoughtful UI designing. This question is likely to be added to future benchmarks.

• Exceptional UI Designing:

  • The model is "insanely good at UI," surpassing even the Gemini 2.5 Pro, which the presenter frequently uses for converting screenshots into UI components. Its UI generation capabilities are described as "something else altogether."

• Tool Calling Functionality:

  • Successful Test: The tool calling capabilities were tested using the "human relay mode from Rue." The first request resulted in an "on point" tool call.
  • Future Potential: While current tool calls are limited in scope, the presenter expressed excitement for its potential integration into the Gemini CLI, especially if it remains free, which could be "literally awesome."

---

Release Expectations and Pricing There is speculation about the release date, with some suggesting it might be "today," but the presenter believes it will be "after a week or something now," possibly "somewhere near the 20th of October." The presenter also noted that another model from a "good AI company" is expected this or next week. A key hope for the X28 model is that its price will be "at or lower than the price of Sonnet" to bring back the "3.5 sonnet days again" in terms of accessibility and value.

---

Synthesis and Conclusion The new X28 Gemini Pro checkpoint represents a significant advancement, demonstrating superior performance in generation quality, consistency, UI design, and even tool calling. Google has "really cooked" with this model, which offers a substantial improvement over previous checkpoints and current market alternatives like Sonnet. The presenter is "pretty stoked" for its release, anticipating it will set a new standard for AI capabilities.