Devstral + FREE MCPs + Unlimited FREE API: This is LOCAL SONNET! The best LOCAL CODER COMBINATION!

Devstrol: A New Open-Source Model for Agentic Coding

Key Concepts:

• Devstrol: A new open-source model by Mistral AI and All Hands AI, specifically trained for agentic coding.
• SWEV: A benchmark for evaluating code generation models.
• Agentic Coding: The ability of a model to autonomously perform coding tasks, including tool use and file management.
• Context Window: The amount of text a model can consider when generating output (Devstrol has a 128K context window).
• Tool Calls: The ability of a model to use external tools or APIs to accomplish tasks.
• R Code/Kilo Code/Klein: Platforms for running and interacting with language models.
• Firecrawl: A web scraping tool used to enhance the model's ability to access and utilize information from the internet.
• MCP Server: A server that integrates various tools and services, such as Firecrawl and Context 7, to enhance the model's capabilities.
• Prompt Adherence: How well a model follows the instructions given in the prompt.
• Rules File: A set of instructions or guidelines provided to the model to ensure consistent and desired behavior.

Introduction to Devstrol

Devstrol is a new open-source language model developed through a collaboration between Mistral AI and All Hands AI (the open-devon project). It was released on the same day as Google's IO event. The model is designed to excel at agentic coding tasks.

Performance and Capabilities

• SWEV Benchmark: Devstrol outperforms all other open-source models on the SWEV benchmark by a large margin.
• Cost-Effectiveness: It is low cost to run.
• Agentic Coding Focus: Devstrol is specifically trained to handle agentic code, which is a strength compared to other models like DeepSeek.
• Performance Relative to Other Models: It scores above Claude 3.5 Haiku, GPT 4.1 Mini, and other models.
• Size and Hardware Requirements: It is a 24 billion parameter model, light enough to run on a single RTX 4090 or a Mac with 32GB of RAM.
• Context Window: It has a 128K context window.
• API Pricing: The API costs around 10 cents per million input tokens and 30 cents per million output tokens.
• Free Tier: Mistral's API offers a free tier with generous limits: one request per second, 500k tokens per minute, and 1 billion tokens per month.

Accessing and Using Devstrol

• Availability: Devstrol is available on Lama and Hugging Face for local download and use. It also has a free endpoint on Open Router.
• Platforms: It can be used on R Code, Klein, and Kilo Code. Kilo Code offers a free $20 credit for trying it out. It can also be used with Olma locally.
• Agentic Task Performance: Devstrol works extremely well with R Code and Klein due to its training on agentic tasks.

Example: Creating a Minesweeper Game

The video demonstrates Devstrol's capabilities by asking it to create a simple Minesweeper game. Unlike most local models (except GLM), Devstrol successfully performs the agentic tasks, creating files and implementing the game. The initial result is functional but lacks styling.

Improving Results with Rules and Tools

The video highlights some issues and provides solutions to improve Devstrol's performance:

• Prompt Adherence: Devstrol is highly prompt adherent, so providing clear instructions is crucial.
• Rules File: The video recommends using a rules file to guide the model's behavior. The rules file should include instructions to:
  • Use Shad CN tailwind and style components minimally.
  • Maintain a task list and use it to implement all tasks.
  • Avoid using a memory bank.
  • Search for documentation through Context 7 MCP server.
  • Search the internet when necessary using Firecrawl.
• Firecrawl Integration: The video emphasizes the importance of using Firecrawl for web scraping.
  • Benefits of Firecrawl:
    • Easy to use and doesn't bog down the system.
    • Gathers clean data from all accessible subpages, even without a sitemap.
    • Parses and outputs content from web-hosted PDFs.
    • Intelligently waits for content to load.
    • Can click, scroll, write, wait, and press before extracting content.
    • Can supply screenshots of websites to the LLM.
  • Firecrawl Pricing: It has a free plan with 500 credits, as well as paid plans.
• MCP Server Setup: The video explains how to set up Firecrawl with the MCP server in R Code, Kilo Code, or Klein by copying the API key and MCP server settings.

Enhanced Minesweeper Example

With the rules and Firecrawl integration, the model is asked to create a new Minesweeper game, searching the internet for cool styles. The resulting game is significantly improved in terms of styling and aesthetics.

Advantages of Small Coding Models

The video emphasizes that small coding models like Devstrol are beneficial for agentic coding because they enhance security and privacy, especially when dealing with confidential code or when internet access is limited.

Conclusion

Devstrol is a promising open-source model that excels at agentic coding tasks. Its performance can be further enhanced by using a rules file and integrating tools like Firecrawl. The model's small size and cost-effectiveness make it a valuable tool for developers who need a reliable and secure coding assistant.