NoSQL DBaaS Comparison 2022: Couchbase Capella vs. MongoDB Atlas

by Carlo GutierrezJuly 14, 2022

Our new report compares the performance of the two systems when running four workloads: update-heavy, short-range scan, pagination, and JOIN.

Table of Contents

Exponential growth of real-time data

Each year, the amount of data that high-load apps operate with, such as e-commerce and social networks, is only exponentially growing. According to a report by IDC, the Global Datasphere, the summation of all digital content, which was 33 ZB in 2018 will increase to 175 ZB by 2025. Additionally, real-time data will double in size, reaching 30% by 2025.

To deal with a constant increase of real-time data, modern apps are now utilizing NoSQL solutions. According to a report by Dataintelo, the global NoSQL market, which was valued at $3.2 billion in 2019, is expected to reach $22.08 billion by 2026.

Compared to relational databases, NoSQL systems provide better performance, scalability, and resiliency needed by modern applications. However, these solutions require a significant amount of time and effort to deploy, configure, manage, and support. To offset this cost, many organizations are relying on NoSQL databases as a service (DBaaS) instead. In this manner, they can benefit from improved performance, while also avoiding the necessity to invest increasing amounts of time and money on cluster support, deployment, and maintenance.

Real-time data to reach 30% of the Global Datasphere by 2025 (Image credit)

Today, we are announcing the results of our latest research that compares the performance of two NoSQL databases as a service: Couchbase Capella (March 2022 release) and MongoDB Atlas v5.0. The evaluation was conducted on three different cluster configurations—6, 9, and 18 nodes—as well as under four different workloads.

Evaluated database systems

Couchbase Capella is a fully managed database as a service. It combines features of a key-value store allowing operations on single documents and acts as a schemaless document store to access the data by querying through SQL++ (SQL for JSON documents).

MongoDB Atlas is a document-oriented NoSQL database. It has extensive support for a variety of secondary indexes and API-based ad hoc queries, as well as strong features for manipulating JSON documents. The database puts forward a separate and incremental approach to data replication and partitioning that happen as completely independent processes.

To provide verifiable results, the benchmark was performed on Amazon Elastic Compute Cloud (EC2) instances. For consistency, the Yahoo! Cloud Serving Benchmark (YCSB) was used to evaluate the NoSQL databases.

The components of the YCSB client (Image credit)

Workload with 100% reads

During the evaluation, one of the scenarios conducted was Pagination Workload, which is a query with a single filtering option, an offset, and a limit. This simulates a selection by field with pagination. This scenario is used for listings, such as e-commerce category pages or search engine results. The scenario was executed under the following settings:

The read ratio was 100%.
The size of a data set was scaled in accordance with the cluster size: 5 million customers (each 4 KB in size) on a 6-node cluster, 25 million customers on a 9-node cluster, and 100 million customers on an 18-node cluster.
The maximum of a query length reached 100 records.
Uniform was used as a query length distribution.
The maximum query offset reached 5 records.
Uniform was used as a query offset distribution.

To speed up query execution on Couchbase Capella, secondary indexes were created for specific fields by which data is filtered. The database provides two index storage modes: memory- and disk-optimized. The latter is the default mode.

Memory-optimized indexes use an in-memory database with a lock-free skip list, which has a probabilistic ordered data structure and, thus, performs at in-memory speeds. Memory-optimized global secondary indexes were created for filtering fields with index replication on each cluster node as shown below.

CREATE INDEX `query1` ON `bucket`(`address`.`country`) USING GSI;

MongoDB Atlas uses mongos instances to route queries and operations to shards in a sharded cluster. If the result of the query is not sorted, the mongos instance opens a result cursor from all cursors on the shards using a round robin method. To improve the performance of the database, an additional secondary index was added to a filtered field as shown below.

db.customer.ensureIndex( { "address.country": 1 } );

The following queries were used to perform Pagination Workload.

Couchbase SQL++	MongoDB Query
SELECT meta().id FROM `bucket` WHERE address.country='$1' OFFSET $2 LIMIT $3	`db.customer.find({ address.country: $1 }, { _id: 1 }) .skip($2) .limit($3)`

Pagination Workload results

On a 6-node cluster, MongoDB Atlas had the lower throughput of 36,392 ops/sec, while Couchbase Capella had 51,952 ops/sec. On a 9-node cluster, MongoDB Atlas slightly decreased throughput with a rate of 35,325 ops/sec. This happened due to the amount of records per shard. On the other hand, Couchbase Capella increased throughput to 60,680 ops/sec. Couchbase Capella performed best on an 18-node cluster with 65,228 ops/sec, while MongoDB Atlas managed a throughput of 29,300 ops/sec.

Throughput and latency for Pagination Workload (Image credit)

Ivan Shyrma

Couchbase Capella demonstrated good results for Pagination Workload, gradually increasing in throughput, as the number of nodes grew. However, latency rose from 3 ms on a 6-node to 6 ms on an 18-node cluster.

“As in previous benchmarks, Couchbase Capella demonstrated better performance than Atlas due to its active-active, all-worker nodes architecture. Capella is also easier to query due to its SQL support. These factors translate to a better price-to-performance ratio in real-world environments.”
—Ivan Shyrma, Data Engineer, Altoros

MongoDB Atlas had the inverse result, as the cluster size increased, the performance of the database declined. This is due to the size of the data set and indexes being doubled, while the cluster size increased by only 50%. The latency for MongoDB Atlas also decreased from 11 ms on a 6-node cluster to 12.4 ms on a 9-node cluster and finally 14.4 ms on an 18-node cluster.

More performance results?

Check out our full report to learn more about how we configured Couchbase Capella and MongoDB Atlas, as well as how both NoSQL databases performed in the evaluation. In addition to Pagination Workload, the report also compared the DBaaS systems across three other scenarios: update-heavy (50% reads and 50% updates), short-range scan (95% scans and 5% updates), and JOIN (with grouping and ordering applied).