Octofriend + GLM 4.5: This AI Coder is ACTUALLY DIFFERENT & AMAZING!

Key Concepts

• AI Coding Agent
• Open Source
• Octo Friend
• Synthetic (formerly GHF Chat)
• LLM API Compatibility (OpenAI, Anthropic)
• Diff Apply Model
• JSON Fixer Model
• GLM-4.5
• Claude Code
• Local LLM Usage
• Agent Description Files (agents MD)
• MCP Servers

Octo Friend: A New Open-Source AI Coding Agent

This video introduces Octo Friend, a new open-source AI coding agent developed by Synthetic (formerly GHF Chat). The agent aims to provide a robust coding assistant, particularly for local model usage, and offers unique features compared to existing solutions like Anthropic's Claude Code.

Key Features and Functionality

• LLM API Compatibility: Octo Friend is compatible with both OpenAI and Anthropic LLM APIs, allowing users to switch between models (e.g., GPT-4, Claude 3 Sonnet/Opus, GLM-4.5, Kimmy) mid-conversation if a model gets stuck. This flexibility extends to local LLMs as well.
• Custom Open-Weights Models: A key differentiator is the use of two custom open-weights models trained by Synthetic:
  • Diff Apply Model: Fine-tuned on Llama 3 8B, this model is specifically trained to fix errors in diff apply formatting, a common issue with open models. This reduces the need for costly re-runs due to formatting errors (e.g., missing curly braces).
  • JSON Fixer Model: This model corrects errors in JSON-based tool calling, further enhancing the reliability of the agent.
• Flexibility and Configuration: Users can choose to host the custom models locally or use them via Synthetic's API (with a free tier). The diff and JSON fixers can also be disabled if desired. Octo Friend supports various agent description file formats (e.g., agents MD, Claude) and can be configured with local LLMs and MCP servers.
• Interface and Usage: The interface is similar to Claude Code, with a prompt box and model display. The menu, accessed via the escape key, allows users to switch models within the same provider or add new models from different providers.

Step-by-Step Usage

1. Installation: Install Octo Friend using the provided command-line instruction.
2. Setup: Run the octo friend command to initiate the setup process.
3. Configuration:
   • Choose whether to use the custom fixing models (diff and JSON). If yes, set up the Synthetic API key.
   • Select the desired LLM (e.g., GLM-4.5 via Synthetic).
4. Interaction: The interface will appear, allowing you to input prompts and receive code suggestions.
5. Model Switching: Use the escape key to access the menu and switch between models or providers as needed.

Example: Mindsetter Game Creation

The video demonstrates Octo Friend's capabilities by asking it to create a Mindsetter game using HTML, CSS, and JavaScript. The agent generates code in a streaming fashion, allowing the user to approve or reject suggestions in real-time, similar to Claude Code. The GLM-4.5 model, combined with the diff and JSON fixers, performs well in this task.

Sponsor Message: Microsass Fast

The video includes a brief advertisement for Microsass Fast, a Next.js boilerplate designed to accelerate the development of MicroSAS and AI side projects. It offers features like Clerk, Stripe, Resend, HostGQL, and AI instructions to reduce hallucinations, claiming to save developers 50+ hours of setup time.

Key Arguments and Perspectives

• Octo Friend offers a more fleshed-out and less buggy experience compared to some other AI coding agents.
• The ability to use custom fixing models, particularly the diff fixer, makes it a promising option for working with smaller, local LLMs.
• The agent's flexibility in terms of model selection and configuration is a significant advantage.

Notable Quotes

• "It works with any OpenAI compatible or anthropic compatible LLM API and allows you to switch models at will mid-con conversation when a particular model gets stuck."
• "What they have done here is that they have trained an 8B model that is specialized to fix wrong formatting of diffs here and improves the performance of multiple models especially for open and smaller models."
• "It is the only one after open code that I have seen to be quite capable and I hope that it evolves even more."

Technical Terms and Concepts

• LLM (Large Language Model): A type of AI model trained on vast amounts of text data, capable of generating human-like text, translating languages, and answering questions.
• API (Application Programming Interface): A set of rules and specifications that software programs can follow to communicate with each other.
• Open Source: Software with source code that is freely available and can be modified and distributed by anyone.
• Diff Apply: A method of applying changes to code by specifying the differences between the original and modified versions.
• JSON (JavaScript Object Notation): A lightweight data-interchange format that is easy for humans to read and write and easy for machines to parse and generate.
• MCP Servers: (The transcript doesn't provide enough context to define this term precisely. Further research would be needed.)

Conclusion

Octo Friend is presented as a promising open-source AI coding agent with unique features like custom fixing models and broad LLM API compatibility. Its ability to work with local LLMs and its flexible configuration options make it a valuable tool for developers seeking a robust and customizable coding assistant. The video highlights its potential to improve the coding workflow, particularly when using open-source models.