Couchbase Capella vs. MongoDB Atlas vs. Amazon DynamoDB vs. Redis Enterprise Cloud

by Alex Khizhniak, Carlo Gutierrez, and Ivan ShyrmaJune 20, 2023

Our new performance benchmark compares the throughput and latency of popular NoSQL DBaaS systems across diverse configurations and workloads.

Table of Contents

The rise of cloud databases

While IDC expects the global datasphere to reach 175 zettabytes by 2025, this tremendous growth brings about new requirements for handling large volumes of information. As data formats become more diverse, traditional databases with their rigid schemas eventually make it difficult to store and query complex objects efficiently and fast enough. As a result, NoSQL systems emerged in response to a rise in the amount of information and the frequency it is stored, accessed, and updated. Adapting to the data-intensive reality, the NoSQL database market is expected to grow from $4.4 billion in 2021 to $36.5 billion in 2029.

At the same time, the overall database management system (DBMS) market is also changing, continues to explode, and is projected to hit $100 billion in 2023. Since 2021, the ongoing growth has been driven primarily by cloud databases as a service (DBaaS), according to this report from Gartner. With cloud DBaaS systems representing 49% of DBMS revenue in 2021, they have been spreading faster than traditional databases lately—accounting for more than 84% of the overall market growth.

Three years ago, Gartner predicted that 75% of all databases would be deployed or migrated to the cloud by 2022. This forecast was corroborated by a 2021 report from IDC, revealing that 75% of relational database deployments were already in private clouds. Analyzing the DBMS market, experts at Gartner stated, “cloud is now the default platform for managing data…on-premises is the new legacy.” However, this shift introduced additional issues.

The NoSQL database market is to reach $36.6 billion by 2029 (Image credit)

In particular, managing data in the cloud is a major problem for 33% of companies, a 2022 survey revealed, while 27% are challenged with the ability to scale to meet demand. The report also noted that 56% of enterprises with database infrastructure in the cloud have not been able to manage it exactly how they want to. Additionally, 77% of digital leaders want database infrastructure to be “invisible”—to avoid struggling with this layer—and 96% expect it to scale as needed.

Besides, organizations adopting NoSQL databases face additional challenges, requiring a significant amount of time and effort to deploy and manage these systems. Some of the considerations include capacity planning, monitoring, as well as proper installation and configuration.

In response to this, NoSQL DBaaS managed by third-party providers have also grown in popularity in recent years. The market of NoSQL databases deployed in the cloud is predicted to reach $24.5 billion by 2025, according to this report. The growth indicates that NoSQL DBaaS could account for the large portion of the overall cloud DBMS market in two years’ time.

Not only adoption—cloud database offerings are gradually maturing, developing beyond basic functionality. “Whereas in the last 2–3 years, the emphasis has been moving to the cloud…this last year has seen a marked maturing of the majority of offerings,” a recent Gartner’s Magic Quadrant report noted.

Performance is an issue when moving and managing data (Image credit)

Still, a 2021 survey from IDC discovered the top challenges of moving structured information between on-premises and public cloud: performance (38%), data volume (33%), and interoperability (33%). Understanding the importance of performance being the #1 issue for many data engineers, we’ve been focusing on benchmarking database latencies and throughput for years.

Today, we are presenting the findings of a new report that focuses on comparing the performance of popular NoSQL DBaaS. (Here’s an official announcement.) This time, we benchmarked Couchbase Capella, MongoDB Atlas, Amazon DynamoDB, and Redis Enterprise Cloud. The evaluation encompassed distinct cluster configurations—3, 6, 9, and 18 nodes—and four kinds of workloads. Below, we reveal the results of the update-heavy scenario.

How we benchmarked NoSQL DBaaS

To ensure the reliability of our findings, we conducted the comparison on Amazon Elastic Compute Cloud (EC2) instances. For consistent evaluation, the Yahoo! Cloud Serving Benchmark (YCSB) was used to assess the performance of the NoSQL databases.

Described in this blog post, Workload A (update-heavy) is one of the core benchmarking scenarios in YCSB. It is a basic key–value workload simulating the typical actions of an e-commerce solution, where large amounts of data are read and updated constantly. With this idea in mind, it fits the goals NoSQL DBaaS were created for.

The components of the YCSB client

The workload was executed with the following settings:

The read/update ratio was 50%–50%.
The Zipfian request distribution was used.
The size of a data set was scaled in accordance with the cluster size: 25 million records (each 1 KB in size, consisting of 10 fields and a key) on a 3-node cluster, 50 million records on a 6-node cluster, 100 million records on a 9-node cluster, and 200 million records on a 18-node cluster.

Couchbase Capella stores data in buckets and collections, which are the logical groups of items—key–value pairs. vBuckets are physical partitions of the bucket data. By default, Capella creates a number of master vBuckets per bucket to store bucket data and evenly distribute vBuckets across all cluster nodes.

Querying with document keys is the most efficient method, since a query request is sent directly to a proper vBucket holding target documents. This approach does not require any index creation and is the fastest way to retrieve a document due to the key–value storage.

Amazon DynamoDB’s read/write capacity for the workload was calculated through experiments. The chosen values have the best balance of read and write capacities based on cost. For each cluster, the following values were used.

3 nodes: 5,020 read and 9,390 write capacities
6 nodes: 6,600 read and 18,650 write capacities
9 nodes: 9,900 read and 27,575 write capacities
18 nodes: 25,000 read and 53,315 write capacities

For MongoDB Atlas, we sharded the collection by _id, using this command:

sh.shardCollection( "ycsb.usertable", { _id: 1 }, false )

The following queries were used to perform Workload A.

	Read	Update
Couchbase Key–Value API	`collection.get(id, $2, getOptions().timeout(kvTimeout))`	`collection.upsert(id, content, upsertOptions().timeout(kvTimeout). expiry(documentExpiry).durability(persistTo, replicateTo))`
MongoDB Query	`db.ycsb.find({_id: $1})`	`db.ycsb.update(` `{ _id: $1 },` `{` `$set: {` `fieldN: $2` `}` `})`
DynamoDB API	`{` `"TableName": "usertable",` `"Key": {` `"_id": "$1"` `},` `"ConsistentRead": "false"` `}`	`{` `"TableName": "usertable",` `"Key": {_id = {S: $1}},` `"AttributeUpdates": {` `$2={Value: {S: $3} }` `},` `"Action": "PUT"` `}`
Redis CLI	`HMGET $1 $2`	`HMSET $1 $2 $3`

Results for the update-heavy workload

On each type of a cluster, Couchbase Capella significantly outperformed the other databases. On a 3-node cluster, it had a throughput of 232,050 ops/sec with a 2.67 ms latency. Couchbase Capella’s performance improved all the way to an 18-node cluster, where it had a throughput of 423,580 ops/sec with less than a 1 ms latency.

As the cluster size increased, the other databases also demonstrated better performance. Each of the databases showed its best performance on an 18-node cluster. MongoDB Atlas achieved 58,290 ops/sec with a 8.13 ms latency, DynamoDB reached 109,350 ops/sec with a 5.1 ms latency, and Redis Enterprise Cloud was close to DynamoDB with 98,700 ops/sec and a 4.15 ms latency.

Throughput results under Workload A on 3-, 6-, 9-, and 18-node clusters

Couchbase Capella stood out with a latency of about 1 ms on 6-, 9-, and 18-node clusters, and a latency of 2.67 ms on 3 nodes. Amazon DynamoDB had a latency of around 6 ms on every cluster type. Redis Enterprise Cloud had a stable latency of about 5 ms, which decreased to 4.15 ms on an 18-node cluster. It also showed significant improvement in throughput only on an 18-node cluster.

The latency of MongoDB Atlas decreased continuously from 25.67 ms on 3 nodes to 8 ms on an 18-node cluster. However, among other databases, MongoDB had the worst results. Nonetheless, it demonstrated the most growth with a cluster’s expansion from 3 to 18 nodes, with a 3x reduction in latency and a 3x increase in throughput.

Latency results under Workload A on 3-, 6-, 9-, and 18-node clusters

Unlike other databases, which demonstrated solid performance, Amazon DynamoDB produced unstable results—due to a high number of failed operations. Across each type of cluster, Amazon DynamoDB had an average of 43–58% of failed operations, with only the 18-node cluster showing results with 25% of failed operations.

Results for DynamoDB under Workload A on 3-, 6-, 9-, and 18-node clusters

The full benchmark report contains additional information, such as extrapolated cost per billion operations for Workload A and other scenarios.

Ivan Shyrma

“The throughput of each database grew constantly depending on the type of a cluster. All databases achieved the throughput limit for each cluster type, except for DynamoDB during read operations on an 18-node cluster. Couchbase Capella demonstrated high throughput growth and clearly outperformed MongoDB Atlas, Amazon DynamoDB, and Redis Enterprise Cloud on each type of a cluster.”
—Ivan Shyrma, Data Engineer, Altoros

Want more results?

For more details on how we configured each NoSQL DBaaS, as well as the comprehensive performance evaluation, check out the complete report. In addition to Workload A, the benchmark encompassess comparisons across three more scenarios. These include read-only (100% read operations), short-range scan (95% scanning and 5% updates), and pagination (utilizing OFFSET and LIMIT filters).

Download the full report here.