Gemma Coder (Fable Distilled): WHAT.. Fable LIKE OUTPUT from a 12B MODEL?

Key Concepts

• Gemma 4 12B Agentic Fable 5 Compose: A fine-tuned version of Google’s Gemma 4 12B Instruct model, optimized for coding, terminal operations, and multi-step agentic workflows.
• Agentic Tasks: Workflows where an AI model autonomously uses tools, reads information, executes code, and verifies its own output.
• GGUF (GPT-Generated Unified Format): A file format for storing models for inference, optimized for CPU/GPU usage via tools like llama.cpp.
• Quantization: The process of reducing model precision (e.g., Q4KM, Q8) to decrease memory usage and improve speed while maintaining performance.
• Tau 2 Bench: A benchmark specifically designed to evaluate an AI's ability to perform multi-step tool-use tasks.

---

1. Model Overview and Performance Claims

The "Gemma 4 12B Agentic Fable 5 Compose" is a specialized local model designed to handle coding and tool-use tasks.

• Performance: The creator claims a 3.5x improvement over the base Gemma 12B model on the Tau 2 bench (scoring ~55% vs. ~15%).
• Trade-offs: The model sacrifices general knowledge (evidenced by lower MMLU Pro scores) to achieve higher proficiency in technical and agentic workflows.
• Limitations: The creator notes that these results are based on a small, local 20-task evaluation and should not be compared directly to frontier models, which can score above 90% on similar benchmarks.

2. Technical Specifications and Quantization

The model is released under the Apache 2.0 license and is available in four primary quantizations:

• Q3KM: ~5.7 GB
• Q4KM: ~6.87 GB (Recommended "sweet spot" for balance of speed and quality)
• Q6K: ~9.11 GB
• Q8: ~11.8 GB

Recommended Sampling Parameters:

• Temperature: 1.0
• Top P: 0.95
• Top K: 64
• Repetition Penalty: 1.1 (Crucial to prevent broken output or repeating strings).

3. Implementation and Deployment

The model can be deployed via two primary methods:

A. Ollama (Terminal-based):

1. Ensure Ollama is updated to support the Gemma 4 architecture.
2. Create a model file and run ollama create Gemma-for-agentic -f Modelfile.
3. Execute via ollama run Gemma-for-agentic.

B. LM Studio (GUI-based):

1. Download the GGUF file and import it into LM Studio.
2. Configure the sampling settings manually.
3. Use the "Developer" section to start a local API server (OpenAI-compatible) to connect with external editors like Zed.

Integration with Zed:

• Zed can discover models via Ollama or connect to the LM Studio API server.
• Context Length: The default is 4,096 tokens; users are advised to increase this to 8,122+ if hardware memory permits.

4. Practical Assessment and Reliability

The reviewer highlights a significant gap between the model's potential and its current stability:

• Strengths: When it functions correctly, it performs well on one-shot coding tasks, refactoring, and small, self-contained code generation.
• Weaknesses: The model is prone to "buggy" behavior, including:
  • Leaked tool-call tokens or channel artifacts.
  • Failure to complete multi-step workflows.
  • Inconsistent output quality.
• Verdict: It is not currently recommended as a "daily driver" for critical development work. It is best suited for experimentation and quick, one-shot technical tasks where the user can manually verify the output.

Synthesis

The Gemma 4 12B Agentic Fable 5 Compose represents an ambitious step toward local, private, and cost-free agentic AI. While it demonstrates impressive capabilities for a 12B parameter model in specific coding scenarios, its lack of consistency and susceptibility to output errors make it a tool for enthusiasts and experimenters rather than a reliable production-grade assistant. Future iterations (such as the hinted V3) will be necessary to address the stability issues currently hindering its practical utility.