NEW GPT-4o UPGRADE: OpenAI's Powerful Coding Model Beats Gemini 2.5 Pro!?

Key Concepts

GPT-4 Omni, Gemini 2.5 Pro, LM Arena Benchmark, coding capabilities, instruction following, freedom (less censorship), multi-turn prompts, debugging, architectural planning, creative idea generation, SVG generation, web app development, Tetris game development, cost efficiency, open-source models.

GPT-4 Omni Upgrade

Announcement and Initial Reaction

Sam Altman, CEO of OpenAI, announced a significant upgrade to GPT-4 Omni on Twitter. The upgrade received less attention due to hype surrounding new image generation capabilities.

Key Improvements

• Complex Instructions: Enhanced ability to handle multi-turn prompts with precision and consistency.
• Coding: Improved debugging and architectural planning, functioning as a "sharper Dev copilot" for solving tough coding problems.
• Creativity: Generates smarter, more original ideas for brainstorming and reasoning-heavy tasks.
• Emoji Reduction: Reduced emoji usage in generations, focusing more on text.
• Freedom: Less censored and more open, addressing previous restrictions on image and text generation.

Access and Availability

Currently available to paid OpenAI users, with free users gaining access in the coming weeks. API access is available via the "chat GPT 4 old latest" preview version.

Benchmark Results and Comparison

LM Arena Benchmark

GPT-4 Omni jumped to the number two spot on the LM Arena Benchmark, surpassing GPT-4.5. It showed a 30-point improvement over January's version and tied for number one on the coding hard prompt benchmark. It outperforms Grok-3 but lags slightly behind Gemini 2.5 Pro.

Side-by-Side Comparison: GPT-4 Omni vs. Gemini 2.5 Pro

A head-to-head comparison was conducted using various prompts to assess the models' generation capabilities.

1. Responsive Web App (HTML, CSS, JavaScript)

• Prompt: Build a responsive web app for tracking monthly income and expenses with features for adding, editing, and deleting transactions.
• GPT-4 Omni: Generated a functional app with a sleek design, dark mode, and expense visualization.
• Gemini 2.5 Pro: Created a responsive front end but lacked functional dark mode and expense visualization. The "Add expense" button was non-functional.
• Result: GPT-4 Omni was deemed superior due to its functional features.

2. TV Channel Generator

• Prompt: Code a TV that lets you change channels with number keys 0-9 and come up with an idea for a channel for all the numbers.
• GPT-4 Omni: Generated multiple channels with different content but lacked the main TV frame.
• Gemini 2.5 Pro: Generated a more visually appealing TV structure with a static frame and diverse animations for each channel.
• Result: Gemini 2.5 Pro was considered better due to its superior visual structure and animation quality.

3. SVG Butterfly

• Prompt: Create an SVG representation of a butterfly with symmetrical wings and simple styling.
• GPT-4 Omni: Successfully generated an SVG butterfly.
• Gemini 2.5 Pro: Also generated an SVG butterfly, with a slightly preferred design.
• Result: Both models passed, demonstrating strong SVG generation capabilities.

4. Tetris Game

• Prompt: Create a Tetris game in a single HTML file.
• GPT-4 Omni: Generated a functional Tetris game.
• Gemini 2.5 Pro: Also generated a functional Tetris game.
• Result: Both models passed, successfully creating a playable Tetris game.

Conclusion and Final Thoughts

The GPT-4 Omni upgrade represents a significant improvement in coding capabilities, instruction following, and freedom. While it competes well with Gemini 2.5 Pro, the speaker still prefers Gemini 2.5 Pro and DeepSeek models due to their cost efficiency and open-source nature (excluding Gemini). The GPT-4 Omni upgrade may be a preview of the upcoming GPT-5 model.