Automatic Read Scaling with AlloyDB Transparent Query Forwarding (TQF)

Key Concepts

• Transparent Query Forwarding (TQF): An AlloyDB feature that automatically reroutes read-only queries from the primary instance to the read pool.
• Read-After-Write Consistency: A mechanism ensuring that forwarded read queries do not return stale data, maintaining data integrity despite the distributed execution.
• Read Pool: A set of secondary database instances designed to handle read-only traffic, offloading pressure from the primary instance.
• Columnar Engine: An AlloyDB component that accelerates analytical queries by storing data in a columnar format rather than row-based.
• Primary Instance: The main database node responsible for handling write operations and mission-critical transactions.

---

Overview of Transparent Query Forwarding (TQF)

Paul Ramsey, a Product Manager at Google, introduces TQF as a solution for high-performance database environments, specifically targeting organizations like the fictional "Cymbal Investments." The feature addresses the challenge of managing mixed read/write workloads in high-frequency environments without requiring complex application-level traffic splitting.

The Problem: Resource Contention

In high-frequency brokerage systems, the primary database instance often faces significant performance degradation during market open due to:

• Mixed Workloads: Simultaneous execution of high-volume trade transactions (writes) and expensive analytical queries (reads).
• CPU Contention: Read queries competing for the same CPU cycles as mission-critical write operations, leading to increased latency for trade executions.
• Operational Friction: Traditional solutions require manual intervention, such as hunting down read-only workloads and reconfiguring application endpoints, or performing extensive code refactoring.

The Solution: How TQF Works

TQF automates the offloading process, allowing the database to manage traffic distribution intelligently:

1. Automatic Interception: AlloyDB intercepts read-only traffic at the database level.
2. Intelligent Routing: Queries are forwarded to the read pool without requiring the application to point to a different endpoint or undergo code changes.
3. Columnar Acceleration: The read pool can leverage the AlloyDB columnar engine to further optimize and accelerate complex analytical queries.
4. Primary Isolation: By moving read traffic, the primary instance is dedicated exclusively to mission-critical write operations, ensuring low latency and high throughput for trade executions.

Performance and Consistency

A critical breakthrough of TQF is its ability to maintain read-after-write consistency.

• Replication Speed: The system utilizes improvements in replication that are reportedly 25x faster than standard PostgreSQL.
• Data Integrity: Because of this high-speed replication, the application does not experience stale data, even when queries are forwarded to a different instance. This is essential for financial applications where data accuracy is non-negotiable.

Practical Application and Benefits

• Efficiency: Eliminates the need for manual traffic management or complex application re-architecting.
• Scalability: Allows infrastructure architects to scale read capacity independently of write capacity by utilizing the read pool.
• Performance: Ensures that volatile trading sessions do not result in system bottlenecks, as the primary instance remains unburdened by analytical read pressure.

Conclusion

Transparent Query Forwarding represents a significant advancement in database management for AlloyDB. By automating the routing of read-only traffic and ensuring strict consistency through high-speed replication, it allows organizations to modernize legacy databases (such as Oracle or SQL Server) while maintaining the performance requirements of high-frequency, mission-critical applications.