Google's Gemini 2.5 Pro: This NEW Google AI Model is THE NEW SOTA AI Model! (Beats 3.7 Sonnet, O3!)

Key Concepts

Gemini 2.5 Pro, AI model, benchmarks, LM Arena, thinking models, accuracy, swe-bench, agentic code evils, context window, Native multimodality, Gemini platform, AI studio, rate limits, ninja tools, AI video generation, image generation, music generation, document chats, playground, Haiku, pattern recognition, HTML, CSS, JS, SVG, Python, game of life.

Gemini 2.5 Pro Launch and Features

Google has launched Gemini 2.5 Pro, their most intelligent AI model to date. This initial release is an experimental version of 2.5 Pro, which Google claims excels on various benchmarks, notably achieving the top position on LM Arena. A key feature of the 2.5 models is their "thinking" capability, allowing them to reason before responding, leading to improved performance and accuracy. Benchmarks show that Gemini 2.5 Pro outperforms both 03 mini and 3.7 Sonet, particularly on the swe-bench, an industry standard for agentic code evaluations, where it scores 63.8% with a custom agent setup. This model, potentially the "nebula" model previously spotted on LM Arena, is now available with a 1 million token context window and Native multimodality. Native multimodality enables the model to process vast datasets and handle complex problems from diverse information sources, including text, audio, images, video, and code repositories. The model is accessible on the Gemini platform and AI studio for free, although the API is heavily rate limited. The knowledge cut off for this model is January 2025.

Ninja Tools AI Platform

The video promotes Ninja Tools, an AI platform that combines various AI models and experiences in one place. It claims to save users over $600 annually compared to individual subscriptions. Ninja Tools provides access to models like Claude 3 .7 Sonet, GPT 40, and Gemini, along with features like AI video generation, image generation, music generation, and document chats. The platform also offers a playground for comparing multiple AI responses simultaneously. Pricing starts at $11 per month, providing 1,000 chat messages, 30 AI image generations, and five music generations. The video includes a coupon code "AI code king2" for an additional 20% discount.

Testing Gemini 2.5 Pro with Questions

The video tests Gemini 2.5 Pro with 13 questions to evaluate its capabilities. Here's a breakdown of the questions and results:

1. Question: Tell me the name of a country whose name ends with Leah give me the capital city of that country as well the answer should be something like Australia and canara
   • Result: Pass (Correct answer)
2. Question: What is the number that rhymes with the word we use to describe a tall plant the answer should be three because it rhymes with tree
   • Result: Pass (Correct answer)
3. Question: Write a Haiku where the second letter of each word word when put together spells simple
   • Result: Pass (Correct answer)
4. Question: Name an English adjective of Latin origin that begins and ends with the same letter has 11 letters in total and for which all vowels in the word are ordered alphabetically
   • Result: Pass (Correct answer)
5. Question: Pattern recognition question (Implied: identifying the next number in a sequence)
   • Result: Pass (Correct answer: 1999)
6. Question: I have two apples then I buy two more I bake a pie with two of the apples after eating half of the pie how many apples do I have left the answer should be two
   • Result: Pass (Correct answer)
7. Question: Sally is a girl she has three brothers each of her brothers has the same two sisters how many sisters does Sally have
   • Result: Pass (Correct answer)
8. Question: If a regular hexagon has a short diagonal of 64 what is its long diagonal
   • Result: Pass (Correct answer)
9. Question: Create an HTML page with a button that explodes confetti when you click it you can use CSS and JS as well
   • Result: Pass (Working code provided)
10. Question: Create a playable synth keyboard using HTML CSS ja
    • Result: Pass (Working code with functional keyboard and sounds)
11. Question: Generate the SVG code for a butterfly
    • Result: Pass (Generated visually appealing SVG code)
12. Question: Write a Python program that shows a ball bouncing inside a spinning hexagon the ball should be affected by gravity and friction and it must bounce off the rotating walls realistically
    • Result: Pass (Working code with realistic physics simulation)
13. Question: Write a game of life in Python that works on the terminal
    • Result: Pass (Functional game of life implementation)

Conclusion

The video concludes that Gemini 2.5 Pro is a significant advancement, particularly in coding capabilities, an area where previous Gemini models struggled. The model's performance is considered superior to 03 mini, Gro 3, and even 3.7 Sonet. The presenter expresses hope that Google will move the model out of experimental mode due to its potential and the limitations imposed by the rate limits. The free access on AI studio is highlighted as a positive aspect. Overall, the presenter is impressed with Gemini 2.5 Pro and considers it a state-of-the-art model.