Why More Context Makes Your Agent Dumber and What to Do About It — Nupur Sharma, Qodo

Key Concepts

• Agentic Reviews: The use of autonomous AI agents to perform complex tasks like code reviews.
• Deterministic vs. Non-deterministic: Moving from rigid, predictable software pipelines to fluid, probabilistic AI agent workflows.
• Context Window/U-Curve Problem: The phenomenon where LLMs focus on the beginning and end of a prompt while ignoring the "middle" context.
• Context Engine: A system that acts as a "bouncer," ranking and filtering information to provide only the most relevant data to an agent.
• Orchestration Paradox: When agents spend excessive tokens and time researching how to solve a problem rather than actually solving it.
• Multi-Agent Architecture: A system of specialized agents (e.g., security, coding, Jira) coordinated by a "Judge Agent."
• 80/20 Hybrid Approach: A strategy where 80% of the process is dedicated to research/discovery (high-reasoning models) and 20% is dedicated to deterministic validation/summarization (restricted, lower-reasoning models).

---

1. The Evolution and Challenges of Agentic Workflows

Nupur highlights the transition from static prompts to complex agentic workflows. A primary technical challenge identified is the "U-Curve" effect: LLMs tend to prioritize the initial and final inputs of a prompt, effectively "purging" or ignoring the middle context. This makes dumping large codebases into a single prompt ineffective.

2. Context Optimization Strategies

To combat the loss of context, the speaker suggests moving away from "dumping" data toward strategic optimization:

• Context Engine: Uses ranking logic and search patterns to feed agents only what is necessary. Scaling this to hundreds of repositories is difficult due to indexing overhead.
• Hierarchical Summarization: Creating summaries for every file and folder. While effective, it requires high upfront LLM processing costs whenever code changes.
• Knowledge Graphs: Ideal for complex logical dependencies across multiple repositories. It requires significant initial developer effort to map but provides high-quality results for complex architectures.
• Iterative Retrieval: Acts like a "library card" system, allowing agents to look deep into specific code sections only when needed. This is cost-effective and requires low developer input.

3. Solving the Orchestration Paradox

Agents often fall into infinite loops of "researching the method" rather than executing the task. The 80/20 Hybrid Approach resolves this:

• 80% Research Phase: Uses high-reasoning models to explore, plan, and discover. This phase is constrained by "hard gates" (e.g., timeout counters or max-retry limits).
• 20% Validation Phase: Uses smaller, deterministic models to summarize results and validate them against the original goal. This phase does not require high reasoning, reducing latency and cost.

4. Multi-Agent Architecture and the "Judge Agent"

Instead of one "mega-agent" that gets overwhelmed, Kodo utilizes a Multi-Agent Architecture:

• Specialized Agents: Individual agents focus on specific domains (e.g., security, code style, Jira compliance).
• The Judge Agent: Acts as the final arbiter. It collects outputs from specialized agents, checks them for consistency, and filters them against the user's specific requirements or historical PR data.

5. Calibration and Feedback Loops

The system ensures relevance through two primary calibration methods:

1. PR History: The system indexes past pull requests to understand organizational standards. If a developer consistently ignores a specific type of feedback, the system adjusts the "weight" of that recommendation.
2. Rule-Based vs. Recommendation-Based: Users can define "Rules" (mandatory compliance/security checks) versus "Recommendations" (suggestions). The system tracks developer acceptance rates to refine future suggestions.

Notable Quotes

• "Context engine is like a bouncer... it tells you this is more important."
• "The orchestration paradox: they look for the method to solve the problem... most of the API tokens are wasted on finding a way to do it rather than doing it."
• "Instead of one big agent, we create issue expert agents... a judge agent helps us to combine these and make one sense out of it."

Synthesis

The core takeaway is that building effective agentic systems requires moving away from "brute-force" context injection. By implementing a multi-agent architecture governed by a judge agent, utilizing an 80/20 hybrid approach to prevent infinite research loops, and employing iterative retrieval for context, developers can create reliable, specialized AI workflows that scale effectively without losing focus on the original task.