Stop Making Models Bigger, Make Them Behave — Kobie Crawdord, Snorkel

Key Concepts

• RL (Reinforcement Learning): A machine learning training method used here to optimize model behavior for specific tasks.
• GRPO (Group Relative Policy Optimization): A specific RL algorithm used to train the model.
• Tool Use: The ability of an AI model to interact with external software (e.g., SQL databases, schema inspectors) to retrieve accurate data.
• Data Quality: The emphasis on expert-curated, high-quality datasets over simply increasing model size.
• Hallucination: The phenomenon where a model generates incorrect or fabricated information when it fails to retrieve actual data.
• Rubrics: A structured evaluation framework used to break down model performance into specific, actionable behaviors.
• FinQA: A specialized environment for financial question-answering tasks involving table queries.

---

1. Main Topics and Objectives

The presentation, delivered by Kobe Crawford of Snorkel AI, challenges the industry trend of "making models bigger" to solve complex enterprise problems. The core argument is that for specific tasks—such as financial analysis—tool use discipline is more critical than raw reasoning capability.

The primary objective was to demonstrate that a 4 billion parameter model could outperform a 235 billion parameter model on financial tool-use tasks by using high-quality data and Reinforcement Learning (RL).

2. Real-World Application: Financial Analysis

The team focused on a financial analysis use case where models must query databases to answer questions like "What is the year-over-year growth rate of YouTube ads revenue?"

• The Problem: Larger models often attempt to "reason" their way through a query without inspecting the environment, leading to hallucinations when they guess non-existent table names.
• The Solution: By fine-tuning a smaller model with RL, the model learned to first discover available tables, inspect schemas, and self-correct when encountering SQL errors.

3. Methodology and Process

The research team followed a structured approach to optimize the smaller model:

1. Expert-in-the-Loop Data Generation: Snorkel utilized domain experts (PhD level and industry veterans) to create high-quality, verifiable datasets.
2. Environment Setup: They built the FinQA environment, a self-contained, reproducible sandbox that allows models to interact with tools without external dependencies.
3. RL Training: Using the GRPO algorithm, the model was trained in a 21-hour session costing under $500.
4. Evaluation: The team compared training regimes (single-table vs. multi-table vs. curriculum learning). Surprisingly, training on single-table data only yielded the best performance, even when tested on harder, multi-table reasoning tasks.

4. Key Findings and Results

• Performance Uplift: The 4B parameter model’s "pass at one" rate (the probability of getting the correct answer on the first attempt) doubled after the RL training.
• Generalization: The model showed significant improvement on complex multi-table reasoning tasks (jumping from 13.9% to 26.6% accuracy) despite being trained primarily on single-table tasks.
• Behavioral Shift: The smaller model demonstrated superior "tool discipline"—it learned to query for table names and schema info before attempting to answer, whereas the larger model skipped these steps and hallucinated.

5. Notable Quotes

• "Sometimes it's more like taking a sledgehammer to crack a walnut." — Referring to the tendency to use massive models for tasks that require specific tool-use behaviors rather than general reasoning.
• "It turned out it wasn't the reasoning that was the issue. It was the tool use." — Highlighting the core discovery of the research.

6. Strategic Insights

• Rubrics for Evals: The speaker advocates for building "rubrics" to break down model responses. Instead of a binary "correct/incorrect" evaluation, rubrics allow developers to identify specific failure modes (e.g., failure to inspect schema vs. failure to perform math).
• Productionization: Smaller models are more viable for enterprise production because they offer lower inference costs, better data control, and the ability to run on-premise, avoiding the security risks associated with external dependencies.

7. Synthesis

The presentation concludes that the "bigger is better" paradigm is often a misallocation of resources. By focusing on high-quality, expert-curated data and applying targeted RL training, organizations can achieve production-grade performance with smaller, more efficient models. The key to success lies in identifying the specific behavioral failure (in this case, lack of tool discipline) and using a rubric-based evaluation to guide the training process.