Research to Reality: Bringing Frontier ML Research to Production - Vaidas Razgaitis, Higharc

Key Concepts

• Research Legibility: The process of making complex ML research prototypes understandable and actionable for software engineers and product managers.
• Research Prototype Taxonomy (RPT) Document: A technical design document used to bridge the gap between research and production.
• Mono Repo Architecture: A centralized repository structure containing isolated, decoupled microservices for AI/ML models.
• Decomposition: The methodology of breaking down monolithic research prototypes into manageable, production-ready pull requests (PRs).
• Stack Diffs: A workflow technique (using tools like Graphite) to manage asynchronous code reviews for complex, multi-layered features.

---

1. The Challenge: Bridging Research and Production

Vitis, a senior research engineer at Hyark, highlights the friction between ML researchers—who excel at exploring frontier AI but often lack experience in production-grade software engineering—and platform/backend engineers, who excel at robust systems but may not understand the nuances of ML research. The goal is to facilitate a productive "baton pass" between these two groups.

2. Three Focus Areas for Velocity

To improve the speed of moving research into production, Hyark utilizes three specific levers:

A. Research Legibility (The RPT Document)

Hyark requires a Research Prototype Taxonomy (RPT) document for every project. This document acts as a "Technical Design Document" (TDD) tailored for ML. It includes:

• Domain Context: Explaining novel data representations (e.g., circulation graphs, latent space representations) to engineers unfamiliar with the specific research domain.
• Business Goal: Defining the "why" and the value proposition of the ML tool.
• Type Safety: Defining the contract between the core product repo and the ML repo.
• Persistence Layer: Mapping out database needs (researchers are encouraged not to over-engineer this, leaving it for software engineers to refine).
• System Architecture: Defining the anatomy of the workflow, including external LLM calls and chaining.

B. Codebase Structure (The Mono Repo)

Hyark maintains a Python-based mono repo consisting of cleanly isolated, decoupled microservices.

• Architecture: A one-to-one ratio of researcher to microservice.
• Gateway Pattern: A central gateway guards requests and routes them to the appropriate microservice within a Docker Bridge network.
• Layered Design: Each microservice follows a standard structure:
  • API Layer: FastAPI applications with routers.
  • Controller Layer: Wraps the business logic.
  • Services Layer: Core business logic, including calls to foundation models or loading model weights via CI/CD.
• Standardization: Every microservice includes metadata, build instructions, Dockerfiles, and dependency lock files (Poetry/UV), ensuring a consistent "skeletal backbone."

C. Decomposition and PR Review

Moving from a prototype to production is treated as a design problem.

• Slicing: The team analyzes the dependency graph of a monolithic prototype to determine how to "slice and dice" it into smaller, logical components.
• Graphite/Stack Diffs: The team uses Graphite to manage asynchronous reviews. This allows different subject matter experts to review specific, tightly decomposed PRs simultaneously without blocking the entire pipeline.

---

3. Diagnostic Frameworks

Vitis suggests evaluating team performance using these diagnostic questions:

1. Legibility: Is it obvious to a new engineer what tasks are required to move a prototype to production? If there is ambiguity, the RPT document needs refinement.
2. Repository: Are there clear templates for new code? If the team is constantly fighting old abstractions, the repository architecture may need to be refactored.
3. Decomposition: Can the team consistently estimate delivery timelines? If not, the issue likely lies in the coordination of the handoff or the underlying repository structure.

---

4. Synthesis

The core takeaway is that bringing AI research to production is a systems and process problem, not just a technical one. By standardizing the documentation (RPT), enforcing a modular microservice architecture (Mono Repo), and utilizing advanced PR decomposition strategies (Stack Diffs), organizations can significantly reduce the friction between research and engineering, ultimately increasing the velocity of feature delivery.