Run MongoDB compatible apps on Firestore (Zero code changes)

Key Concepts

• Cloud Firestore: A serverless, NoSQL cloud database that scales automatically and integrates with Google Cloud.
• MongoDB Compatibility: A new feature allowing Firestore to be accessed using MongoDB-compatible interfaces.
• Serverless: A cloud computing execution model where the cloud provider manages the infrastructure, allowing developers to focus on code without provisioning or managing servers.
• Cloud Run: A managed compute platform that enables you to run stateless containers that are invocable via web requests or Pub/Sub events.
• Datastream: A serverless Change Data Capture (CDC) and replication service for Google Cloud.
• Change Data Capture (CDC): A set of software design patterns used to determine and track the data that has changed so that action can be taken using the changed data.
• IAM (Identity and Access Management): A Google Cloud service that manages access control for Google Cloud resources.

Firestore and MongoDB Compatibility

The video introduces a significant development: the ability to easily swap between MongoDB-compatible databases and Cloud Firestore. Patrick, who leads the engineering team for Cloud Firestore, explains that this compatibility was built from the ground up into Firestore and is now available to Google Cloud and Firebase customers as part of the new Enterprise Edition. This eliminates the need to modify application code when switching databases, as previously required.

Origin and Evolution of Firestore

Firestore originated from Datastore, an integrated database for Google App Engine. The team "broke out" Datastore to create a standalone, modernized, and universally available NoSQL database, which was launched as Firestore in 2017. Its use cases have expanded to include mobile and web apps, collaborative productivity tools, game backends, and LLM context stores. Firestore supports direct connections from mobile and web via real-time SDKs, as well as server-side code running on platforms like Cloud Run or Kubernetes Engine.

Practical Demonstration: Cymbal Direct E-commerce Application

To illustrate the ease of switching, a simple e-commerce application called "Cymbal Direct" is used.

1. Initial Setup: The application initially uses Cloud Run and a MongoDB-compatible database. An order is placed and visible in this database.
2. Switching to Firestore:
   • A Firestore database is prepared.
   • The Firestore connection string is copied into the application's environment variables.
   • A new version of the app is deployed with the updated environment variable.
   • Another order is placed, and this new order is now visible in the Firestore database.
   • This demonstrates that only a connection string change is required, not code modification.

Why Choose Firestore?

Patrick highlights several key advantages of using Firestore:

• Serverless Simplicity and Pay-per-Use: Firestore operates on a serverless model, aligning with the simplicity and cost-effectiveness of platforms like Cloud Run.
• Automatic Scaling: It handles varying loads, from one query per day to 100,000 queries per second, without requiring manual capacity provisioning, server patching, or sharding. Users pay only for actual reads and writes.
• Google Cloud Integration: Seamless integration with Google Cloud provides a unified experience for IAM, networking, and security.
• Powerful Stack: The combination of Cloud Run and Firestore creates a simple, scalable, and powerful application stack.

Performance Under Heavy Load: Load Test Results

A load test was conducted on the Cymbal Direct shoe store to demonstrate Firestore's scalability:

• Peak Performance: The test reached a peak of approximately 40,000 orders per second.
• Firestore Inserts: Firestore successfully handled 40,000 inserts per second, directly correlating with the incoming order volume.
• Cloud Run Auto-Scaling: During the test, Google Cloud automatically scaled Cloud Run instances from 5 to 100 to manage the load without any manual intervention or pre-provisioning.
• User Experience: The median latency for end-users remained around 18 milliseconds even at peak load.
• Cost Control: While auto-scaling was enabled for this test, users can set a maximum number of instances for Cloud Run to control costs.

Data Migration to Firestore

For developers looking to migrate existing data to Firestore, Datastream is recommended.

• Functionality: Datastream is a serverless replication service that handles the initial data copy and then uses Change Data Capture (CDC) to stream real-time changes from the old database to Firestore.
• Benefit: This process allows for migration with minimal application downtime.

MongoDB Compatibility Scope

The MongoDB compatibility feature supports over 200 of the most commonly used MongoDB features, with ongoing development to add support for more. Developers are encouraged to check the documentation for specific details.

Main Takeaways

Patrick summarizes the key benefits:

1. Flexibility: The ease of switching between MongoDB and Firestore in both directions is a significant advantage for developers.
2. Serverless: Firestore's ability to scale automatically with demand, without upfront provisioning, offers operational simplicity and cost efficiency.
3. Unified Management: For Google Cloud users, Firestore provides a single pane of glass for managing both applications and databases.