Coffee Meets Bagel Meets Redis Enterprise to Innovate in the Face of Real Time Performance Challenges

Coffee Meets Bagel is a highly curated online dating service. Since its founding in 2012, the dating app provider has grown to more than three million users around the globe. As Coffee Meets Bagel makes a concerted effort to expand its international presence it relies on Redis Enterprise Virtual Private Cloud (VPC) for lightning-fast performance that seamlessly scales across a number of critical functions—from caching and user analytics to real-time data streaming.

Before adopting Redis, Coffee Meets Bagel was struggling with its implementation of Cassandra. While Cassandra delivered on basic high write volume requirements, it experienced noticeable issues when it came to simultaneous updates and deletions, resulting in partial outages that impacted end-user experience. As a result, Coffee Meets Bagel moved to open source Redis, running as self-managed clusters, to deliver low read latency while simultaneously tolerating high update volume. Although the company initially adopted Redis as delivered via Amazon ElastiCache, as its Redis implementation grew to become an integral part of Coffee Meets Bagel’s production environment, the DevOps team realized that it would be extremely beneficial to have a fault-tolerant, highly available and scalable solution with someone to call when problems arose. With open-source Redis, you’re on your own and there’s a lot of trial and error that takes place, which is not ideal in a production environment.

During off-peak times, Coffee Meets Bagel runs on approximately 600 reserved AWS instances. At peak activity times, typically when match recommendations are being pushed to users, the company’s spot instances can scale to double that, processing upwards of one terabyte of data an hour with high reliability. In this fast-paced environment, Redis Enterprise VPC performs several critical roles, including: High speed recommendations engine at scale. Redis Enterprise provides high-performing in-memory operations that enable Coffee Meets Bagel to generate highly personalized recommendations for its millions of users, and process billions of user matches at high speeds. This capability is delivered with Redis’ versatile data structures: Geo-based sorted sets efficiently fetch users based on profile criteria, location, and distance, allowing recommendations to be delivered in real-time. Bloom filters provide de-duplicated recommendations in a very space efficient way, eliminating the redelivery of previously provided recommendations for each user profile. Sets and sorted sets add items by users by priority, and query and calculate user scores in constant time to fetch the right set of recommendations. Sinter operations help to calculate the number of mutual friends and prioritize matches accordingly, while also saving on network I/O by performing set operations directly in memory. This also simplifies the application code, delegating operations to Redis in the most efficient manner possible. Primary data storage. Redis Enterprise is the primary database for more than two terabytes (and growing) of user information. Asynchronous fault-tolerant priority queuing delivers granular prioritization and schedules tasks to send push notifications in advance using queuing API with a timestamp parameter. These sophisticated queuing capabilities allowed us to eventually decommission our implementation of Celery, an asynchronous task queue, reducing the amount of I/O between operations and simplifying our application stack. Through Redis Enterprise’s in-memory processing and backlog queue, Coffee Meets Bagel is able to store data for later use. Fast data ingest. Redis Enterprise’s data ingest capabilities seamlessly scale to process upwards of one terabyte of data an hour, including over 300 million exchanged messages and more than one billion matches, without the need for worker throttling.

• For Coffee Meets Bagel, the biggest benefit of Redis Enterprise VPC is also the biggest requirement: high performance. Previously, we needed to throttle at three server workers in order to avoid huge CPU spikes, but with Redis Enterprise, we’re up to 18 without any performance issues whatsoever. It’s really sped up our development cycle. Instead of waiting for operational performance to scale for new application features and expansion, we’re waiting for the developers to catch up to performance.
• In addition to high performance, Coffee Meets Bagel has seen many other benefits as a result of its move to Redis Enterprise, including 24/7 support, automated operations, and significant cost savings to name a few.
• Redis Enterprise provides automated scaling, failover, cluster creation, persistence, and high availability, allowing Coffee Meets Bagel to avoid the operational complexities that would otherwise come with maintaining Redis in an enterprise environment. Redis Enterprise’s auto scaling capabilities have eliminated days of manual effort previously required to ensure that workers were readily available for scaling during high-traffic times. In addition, we’ve had no problem achieving 99.99% uptime.
• As Coffee Meets Bagel has migrated its ElastiCache instances to Redis Enterprise, it’s been able to significantly reduce the number of instances needed on AWS due to Redis Enterprise’s extreme scale and high availability.
• Coffee Meets Bagel is moving toward a dual public cloud infrastructure in order to leverage the unique advantages that both AWS and GCP (Google Cloud Platform) have to offer. Redis Enterprise VPC is completely cloud agnostic, offering the same performance, capabilities, and configurations in either public cloud for seamless operational consistency.

• With Redis on Flash acting as a memory extender for older data, we’ve seen AWS costs savings upwards of 40% for the year, with no sacrifice in latency.