Agents in Production: How OpenGov Built and Scaled OG Assist - Gabe De Mesa, OpenGov

Key Concepts

• Effect: A TypeScript library used for robust error handling, logging, tracing, and structured concurrency.
• Agent-to-Agent (A2A) Protocol: An open specification for inter-agent communication and routing.
• Generative UI: The ability for an AI agent to render custom user interface components (like forms) at runtime.
• Human-in-the-Loop (HITL): A safety mechanism requiring manual approval for agent-initiated mutating operations.
• Sandboxing: Ephemeral, isolated environments for executing code and file operations safely.
• Observability/Tracing: Using Effect’s built-in tracing to monitor performance bottlenecks and debug agentic workflows.

---

1. Architecture and Development Framework

OpenGov’s "OG Assist" is an AI-powered assistant integrated into their ERP software suite (budgeting, procurement, permitting).

• The Shift to Effect: The team transitioned from LangGraph to a custom "Effect Native" agent loop. This move provided full agency over the loop, allowing for fine-grained control over logging, tracing, and structured concurrency.
• Dependency Injection: By utilizing the Effect AI package, the team can hot-swap language models, ensuring the system remains flexible as AI technology evolves.
• A2A Protocol: OpenGov adopted the A2A protocol to standardize communication between the front end and back end. By defining "Agent Cards" (containing names, descriptions, and schemas), they created a rigorous contract that ensures consistency across different product suites.

2. Managing Complexity and Safety

• Long Context Management: To handle token limits and context overload, the team implemented rolling summarization. After a set number of messages, the system generates a summary, which is then used for recall in later parts of the conversation, allowing the agent to maintain "memory" of long threads without exceeding model limits.
• Sandboxing: To prevent risks to production systems, agents execute code and file creation tasks within ephemeral, isolated sandboxes that are torn down immediately after the task is completed.
• Human-in-the-Loop: For high-stakes or mutating operations, the agent loop is deterministically interrupted. The UI presents a prompt requiring a human to explicitly "Accept" or "Reject" the action, ensuring the user remains in control.

3. Evals, Observability, and Feedback

• Feedback Loops: The team emphasizes that "shipping is the start, not the finish." They utilize a binary thumbs-up/thumbs-down feedback mechanism to collect user sentiment, which informs future iterations.
• Automated Evals: CI pipelines run automated tests against real completions to verify that the agent correctly triggers tools and follows instructions.
• Tracing: Because Effect provides tracing out-of-the-box, the team can visualize function calls as spans. This allows developers to profile latency, identify bottlenecks, and cross-reference data across services, which is critical for debugging complex agentic systems.

4. Tools and Skills

OpenGov treats "Tools and Skills" as the fundamental building blocks of their agents.

• Implementation: A tool is defined as a function, grouped into a "Toolkit," and registered with the language model.
• Real-World Application: The agent can perform context-aware actions, such as highlighting specific UI elements on a page or generating a PDF report for a user to download.
• Generative UI: The agent can dynamically generate UI components (e.g., a form with specific options) at runtime based on the user's request, creating a highly personalized experience.

5. Internal Developer Velocity

Beyond customer-facing products, OpenGov leverages AI internally to improve engineering workflows. The team utilizes tools like Claude and Cursor to accelerate code reading, writing, and review processes. Gabe de Mesa notes that building internal tools and skills mirrors the work done for customers, creating a virtuous cycle of development efficiency.

---

Synthesis and Conclusion

The success of OG Assist at OpenGov is attributed to a "big bet" on the Effect library, which provided the necessary infrastructure for observability and robust code architecture. By combining a standardized communication protocol (A2A), rigorous safety measures (sandboxing and human-in-the-loop), and a focus on iterative improvement through automated evals, OpenGov has successfully scaled AI agents into a production environment. The core takeaway is that building production-grade agents requires moving beyond simple prompt engineering toward a structured, observable, and safe software engineering framework.