Why your agents need decision traces, not just documents — Zach Blumenfeld, Neo4j

Key Concepts

• Context Graph: A graph-based knowledge structure that stores not only facts and entities but also decision traces, precedents, and causal chains to enable AI agents to make informed, explainable decisions.
• Graph Database (Neo4j): A database that stores data as nodes and relationships, used here as the "knowledge layer" to connect and resolve information for AI systems.
• Decision Traces: Records of past decisions, including the "why" and "how" behind them, allowing agents to learn from historical precedents.
• Graph Embeddings: A technique to convert graph nodes into vectors, enabling similarity searches based on structural relationships rather than just semantic text similarity.
• Agent Memory: A framework consisting of short-term memory (conversation history), long-term memory (resolved entities), and reasoning (decision traces).
• Cypher: The graph query language used to retrieve data from Neo4j.
• Ontology: The formal schema defining the types of entities and relationships within the graph.

---

1. The Role of Context Graphs in AI

Traditional RAG (Retrieval-Augmented Generation) systems focus on retrieving information to answer questions accurately. Context graphs evolve this by providing the "reasoning layer" necessary for agents to make better decisions. By mapping customer information, transactions, policies, and historical decision precedents, an agent can move beyond simple data retrieval to providing actionable recommendations (e.g., "reject" or "accept" a loan) with a clear, explainable rationale.

2. Technical Framework and Methodology

The speaker highlights a structured approach to building these systems:

• Data Ingestion: Data is pulled from systems of record (CRM, support systems, etc.).
• Entity Extraction: The process uses a multi-stage pipeline:
  1. SpaCy/GLiNER: Initial extraction of entities from raw text.
  2. LLM Fallback: Using Large Language Models to handle complex or ambiguous extractions.
  3. Merging/Deduplication: A strategy to resolve entities over time, ensuring long-term memory consistency.
• Hybrid Search: The system combines semantic search (vector similarity on text) with structural search (graph embeddings) to find relevant precedents based on how previous decisions were made.

3. Tools and Developer Experience

Neo4j provides specific tools to lower the barrier to entry for developers:

• create-context-graph: A CLI tool (similar to create-react-app) that allows developers to scaffold a full-stack application (Next.js frontend, backend, and database) with a single command. It supports various frameworks like Pydantic AI, LangGraph, and CrewAI.
• Neo4j Agent Memory Package: An open-source library that provides the API for managing short-term, long-term, and reasoning memory.
• Graph Data Science (GDS): Used to generate graph embeddings, which allow the system to perform vector similarity searches on the graph structure itself.

4. Real-World Applications

• Financial Services: An agent can analyze a loan request by looking at customer history, current policies, and past similar cases to provide a risk score and a justified decision.
• Healthcare: The demo showed an agent retrieving information regarding prescription medications, utilizing a specific healthcare ontology to navigate complex medical data.
• SaaS Integration: The framework supports importing data from tools like GitHub, Notion, Jira, and Slack, allowing the context graph to be populated with real-world organizational knowledge.

5. Notable Quotes

• "What a context graph enables an agent to do is actually give an answer, should you reject, accept, and why."
• "A graph embedding is the same concept [as text embeddings] except those green nodes... we actually embedded those into a vector. And so, what that means is similar decision traces are now going to be able to be looked up by vector similarity."

6. Synthesis and Conclusion

Context graphs represent a shift from static knowledge bases to dynamic, reasoning-capable systems. By integrating structural graph data with LLM-based agents, developers can create AI that is not only more accurate but also inherently explainable. The project is currently open-source and evolving, with a focus on automating ontology generation and simplifying the integration of unstructured data into structured graph formats. Developers are encouraged to use the create-context-graph CLI to experiment with these concepts and contribute to the growing ecosystem of graph-native AI tools.