Demand-Driven Context: A Methodology for Coherent Knowledge Bases Through Agent Failure

Key Concepts

• Demand-Driven Context: A methodology where AI agents "pull" necessary information by identifying gaps in institutional knowledge while attempting to solve specific tasks, rather than "pushing" all available data into the agent.
• Institutional Knowledge Monolith: The fragmented, outdated, or undocumented state of enterprise data (Confluence, Jira, Slack, tribal knowledge) that hinders AI agent performance.
• Context Gap Scanner: A tool/framework that validates existing documentation against real-world work items (incidents/tickets) to identify what is missing, stale, or duplicated.
• Agentic Knowledge Management: Shifting the role of the AI agent from a passive consumer of data to an active manager that curates and documents knowledge as it works.
• Meta Model: A structural map of how business processes, systems, and APIs relate to one another, helping agents navigate complex enterprise environments.

---

1. The Problem: The "Memento" Effect in Enterprise AI

The speaker draws a parallel between the movie Memento—where the protagonist cannot retain memory beyond 15 minutes—and current AI agents. While LLMs excel at reasoning and code generation, they suffer from a lack of institutional knowledge.

• The ROI Gap: Despite 88% of companies using AI, only 6% report significant value creation. The speaker argues this is because agents fail to move the needle on actual business delivery (e.g., Jira tickets remain stagnant).
• Knowledge Distribution: Enterprise knowledge is typically 20% outdated, 20% unreliable, 10% duplicated, and 40% "tribal" (undocumented knowledge held only by employees).

2. The Methodology: Demand-Driven Context

Instead of attempting to feed an agent the entire "monolith" of company data, the speaker proposes a pull-based strategy similar to Test-Driven Development (TDD).

Step-by-Step Process:

1. Task Assignment: Assign a specific work item (e.g., an incident report) to the agent.
2. Gap Identification: The agent attempts the task, fails due to missing information, and generates a "checklist" of required knowledge.
3. Human-in-the-loop: A domain expert provides the missing information.
4. Curation: The agent uses this new information to solve the task and simultaneously updates the knowledge base, effectively "documenting" the tribal knowledge.
5. Iteration: Repeating this cycle across multiple incidents gradually builds a high-quality, curated knowledge base.

3. Technical Implementation & Tools

• Persistence Layer: The speaker advocates for using GitHub repositories as the primary storage for curated knowledge. This allows for version control, PR-based reviews, and conflict resolution when multiple agents or teams contribute.
• Context Gap Scanner: A tool that takes a set of past incidents and compares them against the existing knowledge base. It categorizes documentation as "Clean," "Stale," "Incomplete," or "Missing."
• Token Management: With modern context windows (e.g., 1M+ tokens), the speaker suggests that for most domains (approx. 96k tokens), it is more efficient to feed the curated context directly into the window rather than relying on complex RAG (Retrieval-Augmented Generation) architectures.

4. Key Arguments & Perspectives

• Agents as Knowledge Managers: The speaker argues that we must stop treating agents as mere consumers. By making them responsible for documenting their own findings, the organization offloads the burden of manual knowledge management.
• Pre-Retrieval Curation: The speaker emphasizes that context should be fixed before the retrieval phase. Trying to fix documentation during an active, high-pressure operational incident is inefficient and stressful.
• The "Source of Truth" Conflict: When combining codebases (GitHub) and documentation (Confluence), conflicts arise. The speaker suggests establishing a hierarchy where code is treated as the ultimate source of truth.

5. Notable Quotes

• "If there is an AGI coming, the first AGI will be a coding agent for sure."
• "Nobody is going to come to your company and fix your knowledge base. You have to fix it yourself."
• "We are moving the agent from a consumer to a knowledge manager."

6. Synthesis and Conclusion

The core takeaway is that the bottleneck for enterprise AI is not the LLM model quality, but the quality of the context provided. By adopting a demand-driven approach—where agents identify their own knowledge gaps through real-world tasks—organizations can systematically break down their "knowledge monoliths." This process transforms tribal knowledge into a structured, version-controlled, and reliable asset, ultimately enabling agents to function semi-autonomously and deliver measurable business value.