GLM-4.6 (I got Early Access) V/S Claude 4.5 Sonnet: Which is the best CODING LLM?

Key Concepts

• GLM4.6: A new open-weight coding model from ZI, successor to GLM4.5.
• Claude 4.5 Sonnet: A language model from Anthropic, used for comparison.
• Open-weight model: An AI model whose weights are publicly available.
• Mixture of Experts (MoE): A model architecture where different parts of the model specialize in different tasks.
• Context Limit: The maximum amount of text a model can consider when generating a response.
• Tool Augmented Reasoning: The ability of a model to use external tools to improve its reasoning.
• Agentic Tests: Tests that evaluate a model's ability to act as an agent, performing tasks and interacting with its environment.
• Kilo: A coding framework or environment used for testing.
• Expo: A framework for building universal native apps.
• TMDB API: An API for accessing movie and TV show data.
• Godo: A free and open-source game engine.

GLM4.6 Overview

• Successor to GLM4.5: GLM4.6 is presented as an improvement over the already strong GLM4.5 model, particularly in coding tasks.
• Model Size: It comes in a single variant: a 355 billion parameter Mixture of Experts (MoE) model with 35 billion active parameters. The absence of a "GLM4.6 Air" model is noted as a potential drawback for users preferring local inference.
• Performance Claims: ZI claims GLM4.6 is on par with or better than Claude 4.5 Sonnet.
• Context Window: GLM4.6 has an increased context limit of 200,000 tokens, matching Claude's context limit.
• Capabilities: It supports tool-augmented reasoning, aligns with human preferences (style, readability, role-playing), and improves cross-lingual performance.

Ninja Chat Advertisement

• All-in-one AI Platform: Ninja Chat is promoted as a platform providing access to top AI models like GPT-4o, Claude 4 Sonnet, and Gemini 2.5 Pro for $11/month.
• Features: The platform includes an AI playground for comparing model responses and a mind map generator.
• Pricing: The basic plan offers 1,000 messages, 30 images, and 5 videos monthly. Discount codes "king25" (25% off any plan) and "king40yearly" (40% off annual subscriptions) are provided.

GLM4.6 Testing Results

Raw Performance

• Leaderboard Ranking: GLM 4.6 scores fourth on the leaderboard without reasoning and fifth with reasoning.
• Price-to-Performance: The model is praised for its price-to-performance ratio and open-weight nature.
• Coding Focus: It's considered one of the best open models for coding, surpassing the previous Sonnet model.
• Reasoning Variant: The reasoning variant is recommended only for tasks requiring planning.
• Image Generation: The model demonstrates decent performance in generating images like floor plans, Panda SVGs, Pokeballs, chessboards, and butterfly simulations. The chessboard example is highlighted as particularly successful, with legal moves.

Agentic Performance

• Leaderboard Ranking: GLM4.6 achieves second position in agentic tests.
• Movie Tracker App: The model generates a movie tracker app using Expo and the TMDB API, noted for its good design and animations, despite some font issues.
• Go-based Calculator: It creates a graphical calculator in Go that scales with the terminal size.
• Godo Game Editing: GLM4.6 successfully edits an FPS game in Godo, adding step tracking and a health bar affected by jumping. This is a significant improvement over previous GLM models.
• Open Code Repo: The model fails to answer the open code repo question.
• Overall Assessment: GLM4.6 is declared the best open model for coding based on these tests.

Claude 4.5 Sonnet Testing Results

Raw Performance

• Leaderboard Ranking: Sonnet 4.5 without reasoning tops the leaderboard, followed by Sonnet 4.5 max reasoning and Opus max reasoning.
• Math Questions: The model fails to answer two super hard math questions.
• Image Generation:
  • Floor plan generation is good but not perfect.
  • Panda SVG generation is considered poor.
  • Pokeball generation has quirks in button placement.
  • Chessboard generation has issues with move logic.
  • Minecraft-style 3D generation has floor leveling problems.
  • Butterfly simulation is decent but not the best.

Agentic Performance

• Movie Tracker App: The app is more cohesive than previous Sonnet versions but still struggles with removing the top title bar and hardcoding the TMDB API key.
• Go-based Calculator: The model creates a good-looking calculator GUI in Go, but it's not as responsive to terminal sizing as the GLM model.
• Godo Game Editing: The model writes code for the life bar and jump mechanics but doesn't implement it into the main scene initially, requiring a second prompt. The resulting UI is considered poorly designed.
• Open Code Repo: The model fails to answer the open code repo question.
• Leaderboard Ranking: Sonnet 4.5 scores fifth in agentic tests.

Comparison and Final Thoughts

• GLM4.6 Superiority: The presenter concludes that GLM4.6 is superior to Claude 4.5 Sonnet in general coding tasks.
• Cloud Code Disappointment: The presenter expresses disappointment with Cloud Code due to its high cost and lack of significant performance improvements.
• GLM4.6 as the Preferred Choice: The presenter states that GLM4.6 has become their AI coder of choice due to its open-weight nature, affordability, and responsiveness to developer feedback.
• Anthropic Criticism: The presenter criticizes Anthropic for not reducing model costs despite incremental performance improvements and for potentially "nerfing" Sonnet 4 before releasing Sonnet 4.5.
• Support for Open-Weight Models: The presenter encourages viewers to support open-weight models like GLM to promote competition and affordability in the AI market.

Conclusion

The video presents a detailed comparison between GLM4.6 and Claude 4.5 Sonnet, focusing on their coding capabilities. Through various tests, GLM4.6 emerges as the preferred choice due to its superior performance, open-weight nature, and cost-effectiveness. The presenter expresses disappointment with Claude 4.5 Sonnet and criticizes Anthropic's pricing strategy, advocating for the support of open-weight models. The key takeaway is that GLM4.6 is currently the best open model for coding, offering a compelling alternative to more expensive proprietary options.