ClickHouse > Case Studies > Contentsquare's Successful Migration from Elasticsearch to ClickHouse: A Case Study

Contentsquare's Successful Migration from Elasticsearch to ClickHouse: A Case Study

Technology Category

Cybersecurity & Privacy - Endpoint Security
Cybersecurity & Privacy - Identity & Authentication Management

Applicable Industries

Buildings
Cement

Applicable Functions

Procurement
Quality Assurance

Use Cases

Building Automation & Control
Time Sensitive Networking

Services

System Integration
Testing & Certification

About The Customer

Contentsquare is a SaaS company that provides digital experience analytics. The company helps businesses understand how and why users are interacting with their app, mobile, and web sites. They compute billions of touch and mouse movements and transform this knowledge into profitable actions that increase engagement, reduce operational costs, and maximize conversion rates. Using behavioral data, artificial intelligence, and big data to provide automatic recommendations, Contentsquare empowers every member of the digital team to easily measure the impact of their actions and make fast and productive data-driven decisions to optimize the customer journey.

The Challenge

Contentsquare, a SaaS company, was facing significant challenges with its existing Elasticsearch setup. The company had 14 Elasticsearch clusters in production, each with 30 nodes. However, they were struggling with horizontal scalability, as they were unable to assemble larger clusters and maintain their stability for their workload. This limitation in cluster size meant that they could not handle any tenant that would not fit into a single cluster, severely restricting their ability to grow. The upper bound on the amount of traffic they could handle was slowing down the company's growth for technical reasons, which was unacceptable. They were left with two options: either find a way to host each tenant efficiently in a multi-cluster setup or migrate to a more scalable technology.

The Solution

Contentsquare decided to migrate to a more scalable technology and began looking into OLAP database engines that would meet their requirements of minimal latency for queries, a rich query language, efficiency with spinning disks, and simplicity in deployment and operation. After extensive engineering studies, they found that ClickHouse met all their requirements and began planning the migration. The migration process was divided into three phases: getting familiar with ClickHouse and building a new product with it, mirroring all the existing features with custom tooling to ensure no regression, and migrating their clients one by one. They built a new product on top of ClickHouse to familiarize themselves with the technology, then migrated their main product iteratively, rewriting each API endpoint one by one to use ClickHouse instead of Elasticsearch. Finally, they migrated their customers to the new infrastructure, taking care not to move everyone at once to identify potential issues.

Operational Impact

The migration to ClickHouse has brought significant operational benefits to Contentsquare. The company has been able to overcome the limitations of its previous Elasticsearch setup, enabling it to handle larger volumes of data and traffic, and thus support its growth. The migration process, though complex, was carried out without any regression, thanks to careful planning and a phased approach. The company has also been able to leverage the capabilities of ClickHouse to optimize its query performance, resulting in faster and more efficient data processing. This has not only improved the performance of Contentsquare's applications but also enhanced the experience for its customers, who can now access more historical data and benefit from improved data retention.

Quantitative Benefit