GPT-5 is a freak

GPT5 Unscripted Review

Key Concepts:

• GPT5: OpenAI's latest AI system, a combination of internal models with a smart router.
• Zero-shot learning: Ability to perform tasks from a single prompt without prior training.
• Hallucination: AI generating incorrect or nonsensical information.
• Agent mode: Autonomous operation of the AI to complete a task.
• Benchmarks: Standardized tests to evaluate AI model performance (e.g., AIME, GPQA, SWEBench).
• LM Arena: Platform for blind testing and ranking AI models.
• Confabulations: A benchmark that measures how often an AI hallucinates.

Coding and Application Generation

• Beehive Simulation: GPT5 created a visual simulation of a beehive with hexagonal cells, worker bee paths, and honey storage, including sliders for colony size and resource availability, all within a standalone HTML file.
  • Example: Adjusting colony size increased the number of bees, while decreasing resource availability slowed honey production.
• 3D Racing Game: GPT5 generated a 3D racing game with neon-lit cyberpunk tracks, speed boosts (pink blocks), and collision physics (yellow blocks) in a standalone HTML file.
  • The game was functional "out of the box," demonstrating GPT5's reduced error rate compared to other models.
• Fluid Dynamics Visualizer: GPT5 created an interactive fluid dynamics animation with sliders for diffusion, viscosity, time step, dye amount, vorticity, and dissipation, allowing users to mix multiple color dyes.
  • Example: Adjusting viscosity changed the mixing speed of the dyes.
• Ray Tracing Simulation: GPT5 developed a real-time ray tracing simulation featuring a metallic sphere above a street scene, with adjustable parameters like reflectivity, roughness, metalness, clear coat, and exposure.
  • The model autonomously detected and corrected an initial error, showcasing its bug-fixing capabilities.
• CRM Dashboard: GPT5 generated a CRM dashboard with real-time insights into sales, customer engagement, and marketing campaigns, including interactive graphs and charts.
  • Users could toggle widgets and reposition them within the dashboard.
• Photoshop Clone: GPT5 created a functional Photoshop clone with basic tools like brush, eraser, layers, fill bucket, shapes, text, crop, select, pan, zoom, and move. It also included filters, blend modes, and opacity adjustments.
  • Example: The "select" tool restricted drawing to the selected area.
• Video Effects Page: GPT5 created a page where users could upload a video and apply real-time effects such as grayscale, sepia, invert, brightness/contrast, saturation, hue, sharpen, Gaussian blur, edge detect, RGB split, pixelate, and posterize.
  • The vignette effect was the only one that didn't function correctly.
• Mindfulness Meditation Guide: GPT5 generated an interactive mindfulness meditation guide with calm fractal patterns evolving with breathing exercises and sounds, including timers and progress trackers.
  • Users could adjust the length, breaths, theme (forest, dusk), fractal style (branches, fern), and breathing patterns (478, calm, custom). It also generated binaural sounds.

Location Identification

• Photo Analysis: GPT5 accurately identified the event name (Symphony at Sunset) and location (Sunset Beach Park) from a photo taken at a concert, even without metadata and without a clear view of the main stage.

Science and Research Applications

• Taxonomy Tree: GPT5 created a taxonomy tree of big cat species, displaying their classification from family to genus to species, with hover-over species descriptions.
• Interactive Physics Course: GPT5 generated an interactive course on high school physics with visualizations and animations, including lessons on motion and kinematics, forces and Newton's laws, and energy.
  • Example: The motion and kinematics lesson allowed users to adjust initial velocity and acceleration.
• E-commerce Business Intel Report: GPT5 created a comprehensive business intelligence report on e-commerce growth in Asia from 2020 to 2025, including market size, growth drivers, trends, segmentation, and challenges.
  • The report included citations and a summary table.
• Alexander Disease Report: GPT5 researched and compiled a report on Alexander disease in a 2-year-old patient, including definition, clinical management, research therapies, a summary table, and suggested next steps.
• ACL Injury Rehab Protocol: GPT5 researched rehab protocols and return-to-sport timelines for a 25-year-old athlete with an ACL injury, suggesting preventative training in a sports medicine report with recovery phase graphs.

Image Generation and Agent Mode

• Story Book Generation: GPT5, in agent mode, autonomously generated a five-page story book about a frog who wants to be rich, including images for each page.
  • The frog's appearance was inconsistent across the images, and the page formatting was basic.
  • The image generation used the existing GPT-4o image model.

Hallucination Testing

• Stable Diffusion 5 Test: GPT5 correctly identified that Stable Diffusion 5 does not exist, demonstrating its ability to avoid hallucinating information.

Availability and Usage

• GPT5 is available on ChatGPT for both free and paid users.
• Free users may have limited access to GPT5 and may be using a smaller variant of the model.
• Paid users can explicitly select GPT5.

Specs, Pricing, and Performance

• GPT5 is described as an AI system, not just a large language model, and is a combination of several internal models with a smart router.
• OpenAI claims significant reductions in hallucinations and improvements in writing, coding, and health.
• Benchmarks:
  • AIME: GPT5 Pro achieved 100%, matching Grok-4 Heavy.
  • SWEBench Verified: GPT5 with thinking scored 74.9%, surpassing Claude Opus 4.1.
  • Healthbench Hard: GPT5 significantly outperformed GPT-4o and GPT-3.
  • Hallucination Rate (Healthbench): GPT5 with thinking had the lowest rate at 1.6%.
• LM Arena: GPT5 ranked number one across all categories.
• LiveBench (Abacus AI): GPT5 ranked number one.
• Artificial Analysis: GPT5 (high version) ranked number one.
• Pricing: GPT5 High is priced at $3.4 per 1 million tokens, comparable to Gemini 2.5 Pro and cheaper than Grok-4.
• Creative Writing: GPT5 is ranked number one in creative writing benchmarks.
• Confabulations Leaderboard: GPT5 is ranked number one for minimizing hallucinations.

Conclusion

GPT5 demonstrates significant advancements in coding, reasoning, and reducing hallucinations, particularly in health-related queries. While benchmark improvements over previous models are sometimes marginal, GPT5 consistently ranks at the top of independent leaderboards, showcasing its overall superior performance and cost-effectiveness. The rapid pace of AI model development continues, with new and improved models emerging in short intervals.