AI System Design: From Idea to Production - Apoorva Joshi, MongoDB

Key Concepts

• RAG (Retrieval-Augmented Generation): Augmenting an LLM’s pre-trained knowledge with external, domain-specific data.
• Agentic Systems: Semi-autonomous AI systems that use control flows or tools to complete tasks.
• Guardrails: Mechanisms to detect and mitigate invalid, irrelevant, or harmful inputs and outputs.
• Semantic Caching: Storing previous LLM responses to similar queries to reduce latency and cost.
• Structured Outputs: Ensuring the LLM returns data in a predictable format (e.g., JSON) for programmatic reliability.
• Human-in-the-Loop (HITL): A design pattern where AI provides recommendations, but humans retain final decision-making authority.

---

1. The Framework for AI Development

The speaker argues that in the age of AI, "specs are the new code." Rather than rushing to "ship it," developers should follow a four-phase framework to ensure reliability and business value:

1. Product Requirements: Defining the "what," "who," and constraints.
2. System Design: Selecting data strategies, architectures, and patterns.
3. Evaluation & Monitoring: Establishing metrics for performance before and after deployment.
4. Optimization: Refining for cost, latency, and reliability.

2. Real-World Case Study: Health Insurance Claims Review

The speaker applies the framework to a system designed to assist medical reviewers at "MDB Health."

• The Problem: Manual review takes 2 days (4–12x slower than industry standards), delaying patient care.
• Constraints: Data must remain in an approved cloud; human review is mandatory for complex cases and all denials.
• Success Metric: Reduce processing time for urgent claims from 2 days to 1 hour within 90 days.

3. Data Strategy and Architecture

• Data Sources: Clinical guidelines (PDFs), coverage policies (PDFs), and patient history (MongoDB).
• Processing: Clinical guidelines require chunking and embedding for vector search. Patient history requires PII (Personally Identifiable Information) removal.
• Retrieval Techniques: A combination of Vector Search (for semantic relevance) and Keyword/Metadata filtering (for specific diagnosis/procedure codes) is recommended.
• Architecture: The system follows a Control Flow pattern:
  1. Receive claim/notes.
  2. Retrieve relevant guidelines and history.
  3. LLM generates a recommendation with citations.
  4. If complex or a denial, escalate to a human reviewer.
  5. Log final decision and reasoning in MongoDB.

4. Evaluation and Monitoring

The speaker emphasizes that evaluation must be built in from the start.

• Guardrails: Used to filter out irrelevant inputs (e.g., "write me a poem") and ensure outputs contain required citations.
• Key Metrics:
  • Input/Output Guardrail Compliance: Tracking rejection rates and missing citation rates.
  • Faithfulness: Ensuring the LLM’s verdict is strictly rooted in the retrieved documents.
  • Production Monitoring: Tracking human override rates and the time taken for human review as indicators of system quality.

5. Optimization Techniques

Once the prototype is functional, the focus shifts to production-readiness:

• Accuracy: Use Reranking to ensure the most relevant information is surfaced to the LLM.
• Cost/Latency: Implement Semantic Caching for recurring claim types and Batch Processing for non-urgent requests.
• Reliability: Enforce Structured Outputs to ensure the system consistently provides both a verdict and the necessary citations.

6. Notable Quotes

• "Specs are the new code. The art is in defining the product requirements, the system design, and evaluate criteria so you can be confident that your AI coding buddies are building the right thing."
• "The most common mistake I see is over-engineering the solution before knowing what's actually failing."
• "You can't improve what you can't measure."

Synthesis

The core takeaway is that successful AI application development is not about the speed of coding, but the rigor of the design process. By defining clear business constraints, starting with the simplest possible architecture (Control Flow/RAG), and embedding evaluation metrics from day one, developers can build reliable, high-stakes systems that provide measurable business value while maintaining necessary human oversight.