Migrate from Cosmos DB RU for MongoDB: vCore or Atlas?

In today's cloud-native world, choosing the right database infrastructure is critical for application performance, cost management, and operational efficiency. Microsoft's Cosmos DB has been a popular choice for organizations needing globally distributed, multi-model database services. However, many users have faced challenges with performance and the Request Unit (RU) pricing model, leading to increased interest in alternative solutions.

In this post we explore the migration path from Cosmos DB RU for MongoDB to the newer Cosmos DB vCore for MongoDB option, including common challenges, alternative solutions, and a streamlined migration approach using dsync.

Common Challenges with the RU Model for MongoDB

While Cosmos DB's RU for MongoDB model excels at elastic scalability and point reads, it presents several key challenges:

Unpredictable Pricing: The consumption-based RU model can make costs difficult to forecast, especially for workloads with variable traffic patterns. Many organizations find themselves over-provisioning to avoid performance issues, resulting in higher costs than necessary. Cosmos RU supports auto-scaling that can help to mitigate some of the performance impacts, but it doesn’t scale down to 0 and the lower bound depends on the data size and the maximum allowed RU set.

Performance Limitations: For data-intensive operations like large-scale data ingestion or scan-heavy workloads, the RU model often requires substantial provisioning, which can become prohibitively expensive. It also requires knowing in advance which collections need to be sharded with what shard key.

Limited Change Stream (CDC) Support: Cosmos DB RU only supports creating a change stream for individual collections, and it doesn’t generate change events for delete operations. This is often a major limiting factor for event-driven architectures.

Alternative Options

When considering alternatives to Cosmos DB RU for MongoDB, several options emerge:

Self-Managed MongoDB

Pros:

• Complete control over deployment and configuration

• Potential cost savings for certain workloads

Cons:

• Lacks many enterprise-grade features

• Requires MongoDB-specific operational expertise

• Higher maintenance overhead

  
  

MongoDB Atlas

Pros:

• Advanced features including encryption (with queryable encryption)

• Robust streaming capabilities and vector search functionality

• Excellent performance characteristics

• Native Document engine and feature-rich queries

Cons:

• Limited deployment flexibility (minimum of 3 nodes required)

• Introduces a new vendor into your infrastructure ecosystem if you’re not already using MongoDB

  
  

Cosmos DB vCore for MongoDB (New Option)

Pros:

• PostgreSQL-based architecture

• Open-source engine with available repository

• Integrated vector database capabilities

• Seamless integration with other Azure products and services

• Supports single-node deployment for cost optimization

• Familiar MongoDB compatibility

Cons:

• Not all MongoDB features and query operators are supported

• Some performance optimizations are still a work in progress

  
  

Seamless Online Cosmos DB Migration with dsync

The most straightforward path to migrate from Cosmos DB RU to vCore for MongoDB or MongoDB is using the dsync tool. This approach simplifies and accelerates what would otherwise be a lengthy and complex migration process. What would take many hours with dump-restore, now takes minutes!

Basic migration syntax:

dsync $COSMOS_RU $COSMOS_VCORE

You can find additional details, examples and configuration options in dsync documentation.

Developed by Adiom, dsync is an open-source tool designed to make online NoSQL database migrations remarkably simple for developers and DevOps teams. Inspired by the simplicity of the familiar "rsync" tool for file operations, dsync brings that same ease of use to online database migrations.

Key Features of dsync:

• Lightweight: Minimal resource requirements

• Highly Parallelized: Optimized for performance during migrations

• Reliable: Includes resumability and robust data validation checks

• Portable: Runs anywhere - from your laptop to a VM or Docker container

• User-Friendly: Zero learning curve for anyone familiar with basic bash commands

  
  

The dsync tool automates both initial data synchronization and change data capture (CDC), managing the coordination and transition between these phases without manual intervention.

Migration Considerations

Even when migrating between MongoDB-compatible APIs, several factors require special attention:

Indexes

Ensure your index strategy is properly transferred and optimized for the new environment. Different MongoDB implementations have subtle differences in indexing behavior.

Performance Validation

Thoroughly test query and write performance under your expected load patterns. The vCore model's performance characteristics will differ from your RU-based deployment.

Data Integrity

Decide what approach to use for validating data integrity post-migration. This is especially critical for online migrations. Our CEO's post on Medium has a good summary of common methods. Dsync already includes embedded validation checks based on counts and document hash (docs).

NOTE for Cosmos vCore destinations: the collection.estimatedDocumentCount() method may return incorrect counts after migrating a large data set. We recommend using using the count() or countDocuments() methods that return accurate information but may take a little longer.

Delete Operations

Note that Cosmos DB doesn't expose deletes in its change stream, which can present a challenge during migration if you’re looking to minimize downtime (so-called live or online migration). Ideally, you should pause or queue delete operations at the application level during migration. If that's not possible and you are migrating to Cosmos DB vCore for MongoDB, the Adiom team offers specialized support.

Best Practice

For a successful migration, replicate the entire migration process across all environments, starting with development moving through your testing environments before tackling production. This approach helps identify and resolve any migration-specific issues early in the process.

The portability of dsync makes it particularly well-suited for this progressive migration strategy, allowing you to establish and refine your migration playbook with minimal effort.

Conclusion

As organizations look to optimize their database infrastructure for performance and cost, the migration from Cosmos DB RU to vCore for MongoDB represents an attractive option. With the right tools and approach, this migration can be executed with minimal disruption to your applications and operations.

The dsync tool simplifies this journey, bringing rsync-like simplicity to what would otherwise be a complex database migration process. Whether you're looking to reduce costs, improve performance for specific workloads, or gain more deployment flexibility, the new vCore option deserves serious consideration.

If you need assistance with a migration, help with evaluating options, or simply want more information about dsync, please reach out to the our team as we are continuing to enhance this open-source tool for the benefit of the broader database community.