Ternary's Innovative Approach to Managing Customer Generated Data at Scale

Ternary is a software company that provides a FinOps platform to serve Google Cloud (GCP) customers' unique needs. The platform empowers cloud engineers, IT finance, and business teams to collaborate more effectively on public cloud cost optimization. The company is in high-growth mode on all fronts, and its platform now supports many of the largest GCP customers. These customers bring the challenge of providing a SaaS platform at scale. Ternary is based in San Francisco, CA, United States and has between 11-50 employees.

Ternary, a FinOps platform provider for Google Cloud (GCP) customers, was facing the challenge of managing and analyzing the large volume of cost-related data generated by its rapidly growing customer base. The platform, which aids cloud engineers, IT finance, and business teams in optimizing public cloud costs, had to deal with the complexities of providing a SaaS platform at scale. The challenge was to break down costs by projects and other dimensions across a time series for users with many values in a given dimension. The company was frequently running into issues with Cube’s response limit of 50,000 rows, which could result in incomplete datasets and inaccurate total cost calculations. The challenge was to present complete, accurate data to users, enabling them to perform multidimensional analysis of vast volumes of cost data.

Ternary developed a solution that focused on presenting the 'most interesting' data points to users, defined as their highest spend in a 30-day period and the general shape of their expenses during that time. To avoid exceeding Cube's response limit, Ternary imposed limits on their queries so that users only saw this data. This made their charts more readable and eliminated the concern about maxing out the Cube response limit. However, this approach meant that some data was inevitably lost. To account for this, Ternary rolled all the 'less interesting' data points into an 'other' column, ensuring the accuracy of total spend and subsequent calculations. Ternary used a 3-query approach to expose the 'most important' data and generate the 'other' column. This approach involved requesting all cost totals day by day, requesting the top N number of the 'most interesting' combinations of projects and categories, and then requesting cost day by day broken down by project-category pairs using filters generated from the second query.

• The 3-query approach developed by Ternary has significantly improved the company's ability to manage and analyze large volumes of customer-generated data. By focusing on presenting the 'most interesting' data points to users, Ternary has been able to make their charts more readable and eliminate concerns about exceeding Cube's response limit. The inclusion of an 'other' column to account for 'less interesting' data points has ensured the accuracy of total spend and subsequent calculations. This approach has also allowed Ternary to stay within the 50K-row limit while providing a complete overview of the most expensive projects and preserving the total cost. The solution has proven to be cost-effective, with the benefits outweighing the potential increase in costs associated with the increased number of Cube queries.

• The 3-query approach allowed Ternary to manage large volumes of data without exceeding Cube's 50,000 row limit.

• The duration of a request where N = 20 using the 3-query approach was the same as that of using one query to gather all the data.

• The 3-query approach allowed Ternary to present complete, accurate data to users, enabling them to perform multidimensional analysis of vast volumes of cost data.