Windsurf SWE-1, Lite & Mini: Windsurf basically made DISTILLED SONNET for LOW-COST?!

Key Concepts

• SWE Models (SWE1, SWE1 Light, SWE1 Mini): Windsurf's proprietary AI models optimized for software engineering.
• Fine-tuning vs. Training from Base: The distinction between adapting an existing model versus creating one from scratch.
• Context Length: The amount of information the model can consider when generating responses.
• Knowledge Cutoff Date: The date up to which the model's training data is current.
• Benchmarks: Standardized tests used to evaluate the performance of AI models.
• Tool Call Reasoning: The ability of a model to understand and utilize external tools or APIs.

1. Introduction to Windsurf's SWE Models

• Windsurf has launched its own AI models called the SWE models (SWE1, SWE1 Light, and SWE1 Mini).
• These models are exclusively available within the Windsurf environment.
• The models are not open source.
• They are positioned as Wave 9 updates, optimized for the entire software engineering process, not just coding.

2. SWE Model Flavors and Availability

• SWE1:
  • Claims to have approximately Claude 3.5 Sonnet levels of tool call reasoning.
  • Cheaper to serve than other models.
  • Available to all paid users for a promotional period at zero credits per prompt.
• SWE1 Light:
  • A smaller model that replaces Cascade Base with improved quality.
  • Available for unlimited use to all users, both free and paid.
• SWE1 Mini:
  • A small, extremely fast model.
  • Powers the Windsurf tab passive experience (autocomplete, next prediction).
  • Not directly accessible for chat.

3. Training Data and Methodology

• The models are trained on Windsurf's own data.
• The training recipe encapsulates incomplete states, long-running tasks, and multiple surfaces, leveraging insights from the heavily used Windsurf editor.
• A new data model called the shared timeline was used.

4. Benchmarks and Evaluation

• The presenter criticizes the provided benchmarks as being closed source and uninformative.
• The "conversational SWE benchmark" is highlighted as particularly problematic because the questions are not disclosed.
• The presenter suggests using standard benchmarks like swbench for better evaluation.

5. Practical Testing and Code Generation

• The presenter upgrades Windsurf to the latest version to access the new models.
• Both SWE1 and SWE1 Light are currently unlimited to use.
• The presenter believes the models are fine-tuned versions rather than trained from scratch.
• The SWE1 model's knowledge cutoff date is June 2024, which is considered recent.
• The model allows image attachments, but it cannot process them.
• A simple car driving game in 3JS is used as a test case.
• SWE1 generates the code quickly, but it contains errors.
• Cascade (presumably another Windsurf model) is used to fix the errors.
• The corrected code works, but the presenter notes that the initial code used CDNs for libraries despite initializing a 3JS project, which is considered inefficient.
• The presenter finds the model not as good as Gemini 2.5 Pro or Sonnet.

6. Limitations and Potential Improvements

• The code generation quality degrades with complexity.
• The knowledge cutoff date can be a limitation.
• SWE1 Light is less effective than SWE1 and produces more errors.
• The presenter suggests that these models might be fine-tuned versions of OpenAI models, given the merger between Windsurf and OpenAI.

7. Conclusion and Future Prospects

• The presenter concludes that the models are likely not pre-trained from scratch.
• The models may be worth using within Windsurf due to their current free availability and potential cost savings.
• The presenter hopes for improvements to the models, making them more robust and potentially unlimited for the $15 membership tier.
• Local availability of the light variant, even in a proprietary format, is suggested for offline use.
• The presenter expresses interest in API access to the models.

Notable Quotes

• "SWE1 has approximately clawed 3.5 sonnet levels of tool call reasoning while being cheaper to serve."
• "Enabled from the insight from our heavily used Windsurf editor we got to work building a completely new data model the shared timeline and a training recipe that encapsulates incomplete states longunning tasks and multiple surfaces."

Technical Terms and Concepts

• CDN (Content Delivery Network): A network of servers that delivers web content to users based on their geographic location.
• 3JS: A JavaScript library used for creating and displaying animated 3D computer graphics in a web browser.
• Fine-tuning: The process of taking a pre-trained model and further training it on a specific dataset to improve its performance on a particular task.
• Pre-training: Training a model on a large dataset before fine-tuning it for a specific task.

Logical Connections

• The video begins by introducing the SWE models and their different versions.
• It then discusses the training data and methodology used to create these models.
• The presenter critiques the benchmarks provided by Windsurf and conducts practical tests to evaluate the models' performance.
• The limitations of the models are discussed, along with potential improvements and future prospects.
• The video concludes with an overall assessment of the models and their potential value to Windsurf users.

Synthesis/Conclusion

Windsurf's SWE models represent an attempt to integrate AI deeply into the software engineering workflow. While the models show promise in terms of speed and cost-effectiveness, they also exhibit limitations in code generation quality, knowledge cutoff date, and benchmark transparency. The presenter suggests that these models are likely fine-tuned versions of existing models rather than being trained from scratch. Despite their shortcomings, the SWE models may be worth exploring for Windsurf users, especially given their current free availability. Future improvements and potential integration with OpenAI models could further enhance their value.