DeepSeek Just Updated—The Best Coding Model Yet?

Deepseek V3 Minor Update Summary

Key Concepts:

• Deepseek V3: A coding-focused Large Language Model (LLM).
• Token Generation: The process of the LLM producing text or code.
• Context Window: The amount of text the model can consider when generating output.
• Reasoning vs. Non-Reasoning Models: LLMs that can solve complex problems vs. those that primarily focus on pattern recognition.
• Prompt Engineering: Crafting effective prompts to guide the LLM's output.
• Open Router: An API platform providing access to various LLMs.
• Quantization: A technique to reduce the size of a model, potentially impacting performance.

Coding Capabilities

• Impressive Coding Performance: The Deepseek V3 minor update demonstrates significant improvements in coding capabilities.
• One-Shot Website Generation: The model successfully generated a complete, functional website from a single, simple prompt. The prompt was: "code a modern landing page using HTML CSS JS and put everything in a single file".
• Token Output: The website generation resulted in almost 20,000 tokens.
• Bouncing Ball Example: The model created an HTML script for a bouncing ball inside a spinning tessaract.
• Interactive Updates: The model was able to modify the code to highlight the side of the tessaract the ball was touching, demonstrating its ability to understand and implement specific instructions.

Reasoning Abilities

• Reasoning Improvement: The updated model shows improved reasoning capabilities, even though it's not explicitly a reasoning model.
• "Read and Rewrite" Prompt Technique: The technique of having the model rewrite the user input verbatim before answering significantly improves its reasoning performance.
• Trolley Problem Example: When presented with a modified trolley problem where the people on the track are already dead, the model initially answered incorrectly. However, after applying the "read and rewrite" technique, it correctly identified that the people's state of being dead changes the ethical considerations.
• Schrödinger's Cat Example: Similarly, the model correctly answered a modified version of the Schrödinger's cat problem after rewriting the prompt, recognizing that a dead cat remains dead.
• Jug Problem: The model provided a concise and correct solution to a classic water jug problem (measuring 6L using 6L and 12L jugs).
• Monty Hall Problem Failure: The model failed to correctly solve a modified version of the Monty Hall problem, indicating limitations in its reasoning abilities.
• Farmer's Paradox Failure: The model also struggled with a modified version of the farmer's paradox, generating unnecessary steps.

Availability and Access

• Deepseek Website: The updated model is available on the Deepseek website. Accessing Deepseek R1 defaults to the new model.
• Hugging Face: The model weights are available on Hugging Face, but require significant storage (700GB) and GPU capacity to run locally.
• Open Router API: The model is accessible via the Open Router API under the name DeepSQ3 0324.
• Free API Access: Open Router offers free API access with a flexible token allowance.
• Token Limit: Open Router allows for a max output context of 131,000 tokens, significantly higher than the 8,000 token limit mentioned in the Deepseek documentation.
• Potential Quantization: The Open Router version might be a quantized version of the model, potentially affecting performance compared to the original.

Performance and Future Expectations

• Minor Upgrade Impact: The improvements observed are from a minor update, suggesting potentially significant advancements in future versions (R2, V4).
• Token Generation Speed: The model exhibits good token generation speed.
• Image Generation: Image generation capabilities are present but not particularly impressive.
• Official Benchmarks: The speaker is awaiting official benchmark results and chatbot arena leaderboard rankings.

Conclusion

The Deepseek V3 minor update represents a notable advancement in coding LLMs, showcasing impressive one-shot website generation and improved reasoning capabilities when combined with effective prompting techniques. The model's availability through various platforms, including the Deepseek website and the Open Router API, provides accessible options for experimentation and application. The high token output capacity on Open Router is particularly beneficial for complex software development tasks. While limitations remain in certain reasoning scenarios, the observed improvements from a minor update suggest a promising trajectory for future Deepseek LLM releases.