NoSQL DBaaS Comparison 2021: Couchbase Capella vs. Azure Cosmos DB

by Carlo GutierrezOctober 29, 2021

Our new report compares two NoSQL databases as a service (DBaaS) while processing update-heavy, short-range scan, pagination, and JOIN workloads.

Table of Contents

Reducing the complexity of NoSQL systems

NoSQL systems provide superior performance and better scalability compared to traditional relational databases. While NoSQL systems can help to achieve the performance required by workloads today, they also entail significant effort and time for deployment, configuration management, and support. Performance drops caused by lag or other factors can lead to major issues and serious troubleshooting.

To reduce the complexity associated with NoSQL systems, databases as a service (DBaaS) emerged as a fully managed solution deployed in the cloud. NoSQL DBaaS providers manage these solutions, removing the need for their customers to perform administrative tasks, such as securing, scaling, monitoring, maintaining, and updating the database. Customers are simply charged according to their usage of the service, enabling them to optimize resources, while benefitting from NoSQL systems.

NoSQL market projection (Source)

Last year, we compared the performance of three NoSQL DBaaS systems: Couchbase Capella (then Cloud), MongoDB Atlas, and Amazon DynamoDB. Since a handful of alternatives is available on the market, we expanded our testing to include new offerings. Today, we are announcing the results of our latest benchmark, which compares the performance of Couchbase Capella and Azure Cosmos DB. The evaluation was conducted on three different cluster configurations—6, 9, and 18 nodes—as well as under four different workloads.

The testing environment

Couchbase Capella is a fully managed DBaaS that can be configured through a control panel, which includes a variety of options for modifying clusters, such as instance sizes, configurations, and quantities. Additionally, the DBaaS system can be fine-tuned to deploy specific services to a single or more nodes in the cluster.

Azure Cosmos DB is a fully managed NoSQL database with single-digit millisecond response times, as well as automatic and instant scalability. As a fully managed service, there is no need to worry about hardware provisioning, setup and configuration, replication, software patching, or cluster scaling.

To ensure the verifiable and repeatable evaluation results, we utilized the Yahoo! Cloud Serving Benchmark (YCSB)—an open-source specification and program suite for evaluating retrieval and maintenance capabilities of computer programs. For Couchbase Capella, the benchmark was performed on Amazon Elastic Compute Cloud (EC2) instances. The YCSB client was deployed to four compute-optimized large instances. Each client instance of YCSB produces threads from 25 to 175 in 25-thread increments. This means the total load on the database ranged from 100 to 700 threads with increments of 100 (four clients with 25 threads each) during each test.

The YCSB architecture deployed

YCSB clients running benchmarks for Azure Cosmos DB were spun up on Azure Virtual Machine instances. Client settings for the number of threads remain the same.

Workload with 50% reads and 50% updates

One of the scenarios was an update-heavy workload that simulated typical actions performed on an e-commerce website: 50% reading operations and 50% updates. This is a basic key-value workload executed with the following settings:

The read/update ratio was 50%–50%.
The Zipfian request distribution was used.
The size of a data set was scaled in accordance with the cluster size: 50 million records (each 1 KB in size, consisting of 10 fields and a key) on a 6-node cluster, 100 million records on a 9-node cluster, and 200 million records on a 18-node cluster.

The update operation for Azure Cosmos DB was performed using a stored procedure, which sped up the update query. Azure Cosmos DB uses indexes which have different types, including range, spatial, and composite. By default, Azure Cosmos DB uses range type indexes for each field. To further speed up operations, we only used the ID field during testing. Read and write operations for Couchbase Capella were performed via a key-value API.

The following queries were used to perform the update-heavy workload.

Query	Couchbase Capella N1QL	Azure Cosmos DB
`READ`	`collection.get(id, getOptions() .timeout(kvTimeout))`	`container.readItem(id, partitionKey, ObjectNode.class)`
`UPDATE`	`collection.upsert(id, content, upsertOptions() .timeout(kvTimeout) .expiry(documentExpiry) .durability(persistTo, replicateTo))`	`container.getScripts() .getStoredProcedure("update") .execute(updateArgs, options);`

On a 6-node cluster, Azure Cosmos DB had a throughput of 6,640 ops/sec, while Couchbase Capella had 42,404 ops/sec. On an 18-node cluster, the throughput of Azure Cosmos DB and Couchbase Capella increased to 16,608 ops/sec and 169,632 ops/sec, respectively.

Throughput and latency for the update-heavy workload

The throughput for both Couchbase Capella and Azure Cosmos DB grew as the cluster size increased. However, Couchbase Capella had higher throughput growth and a lower latency of a millisecond.

More performance results?

To learn more about how we configured Couchbase Capella and Azure Cosmos DB, as well as how both performed in the evaluation, check out the full comparison. Besides the update-heavy workload and the pagination workload, the study also compares the DBaaS systems across three other scenarios: short-range scan (95% scans and 5% updates), pagination (100% reads, filtering with OFFSET and LIMIT), and JOIN (with grouping and ordering applied).

Download the full report here.