IMVU's Transformation: Leveraging AWS for Advanced Analytics and Data Streaming

IMVU is the world’s largest avatar-based social network, with over 7 million users monthly. Users can customize their avatars, chat with friends, shop, hang out at cool parties, and earn real money by creating virtual products. The company was one of the pioneers and early adopters of Apache Hadoop, taking advantage of Big Data technologies before they became mainstream. IMVU was looking to enhance and re-architect their platform by augmenting it through advanced analytics and data streaming. The company sought to modernize their platform’s data architecture by introducing CI/CD, Infrastructure as Code (IaC), and other best practices, to achieve faster analytics iterations, better maintainability, and lower TCO.

IMVU, the world’s largest avatar-based social network, was facing challenges with its on-premises infrastructure which was limiting its capacity for advanced analytics. The company had been an early adopter of Apache Hadoop and Big Data technologies, but found it difficult to support and upgrade their 90-node Hadoop cluster and in-house built tooling. IMVU’s analysts lacked the tools to rapidly generate a range of business-critical reports on customer in-game behavior at scale. They were working with historical data in batches, which made analytics more complex and created multiple bottlenecks. Late reports resulted in inaccurate assumptions about customer in-game purchases, leading to slower sales and loss of profit. The analytics team also lacked a test environment to efficiently check analytics assumptions. The company sought to modernize their platform’s data architecture by introducing CI/CD, Infrastructure as Code (IaC), and other best practices, to achieve faster analytics iterations, better maintainability, and lower TCO.

IMVU partnered with Provectus and AWS to drive innovation, implement streaming architecture, enable advanced analytics, and build several AI-powered apps to improve customer retention in the long term. They developed a comprehensive migration and modernization strategy towards NextGen architecture. Their data pipelines were modernized and re-architected to meet the requirements for a modern Data Platform, utilizing open source solutions and AWS services. The Data Platform was designed to use Airflow for both job scheduling and monitoring, to run on Amazon EKS. Data pipelines were optimized to utilize Amazon’s EMR Autoscaling policies. Provectus optimized Hive/Spark jobs, decoupled compute and storage layers, introduced a new query engine based on Apache Presto and implemented concise and self-documented delivery pipelines. The Business Intelligence (BI) layer was optimized to serve the needs of IMVU’s data analytics team. As part of data platform modernization and re-architecture, Provectus migrated Hadoop clusters to Amazon EMR with Data and Compute decoupling.

• IMVU received a scalable, cloud-native infrastructure for its data platform, which unlocked the full potential of AWS cloud and IMVU unique user generated data and created a foundation for near real time analytical and machine learning use cases. The IMVU team can now take advantage of vast amounts of data generated by users to look into customer lifetime behavior and generate a range of comprehensive reports, including but not limited to in-game purchases, customer engagement, and avatar preferences. The analysts of IMVU can also generate reports much faster, whenever they need to. Infrastructure-wise, IMVU has overcome significant technological barriers to fast and seamless code deployments. By having their platform re-architected for AWS, IMVU has created a solid foundation for their cloud-based products, as well as introduced the potential for using Artificial Intelligence and Machine Learning for customer retention.

• 400 Spark Jobs

• 2x TCO Optimization

• 1PB Processed Daily