实例探究 > Global Manufacturer Averts Data Swamp with New Data Lake Architecture

Global Manufacturer Averts Data Swamp with New Data Lake Architecture

公司规模
1,000+
产品
  • Hadoop
技术栈
  • Hadoop
  • Avro
  • RC file
  • Snappy
  • bzip2
实施规模
  • Enterprise-wide Deployment
影响指标
  • Customer Satisfaction
  • Digital Expertise
  • Productivity Improvements
技术
  • 分析与建模 - 大数据分析
  • 应用基础设施与中间件 - 数据交换与集成
  • 基础设施即服务 (IaaS) - 云存储服务
适用行业
  • 电子产品
适用功能
  • 商业运营
  • 销售与市场营销
服务
  • 软件设计与工程服务
  • 系统集成
关于客户
A multi-billion dollar global manufacturer of electronic components, connectors, and sensors wanted to enhance the value being derived from their extensive data. The company had launched a strategic initiative to utilize data as a strategic asset to transform the business. The company had built a Hadoop data lake consisting of multiple disparate data sources of structured and unstructured data, yet they were unable to effectively leverage the data to create actionable business insights. The Hadoop implementation was also showing signs of performance issues. The organization turned to Antuit’s team of big data architects and engineers to improve the performance of their architecture and create a scalable platform for data consumption.
挑战
A global electronics component manufacturer launched a sales and marketing data hub, only to find that it could not scale to handle the volume or velocity that its one terabyte of data presented. They needed an infrastructure that was easier, faster, and more reliable. The company had built a Hadoop data lake consisting of multiple disparate data sources of structured and unstructured data, yet they were unable to effectively leverage the data to create actionable business insights. The Hadoop implementation was also showing signs of performance issues. The organization turned to Antuit’s team of big data architects and engineers to improve the performance of their architecture and create a scalable platform for data consumption.
解决方案
Working collaboratively with the client, the Antuit team audited the existing process, and then re-engineered and implemented a robust scalable architecture. As a result of this process, Antuit uncovered a number of challenges. The client’s existing architecture could not handle the +10 years of sales and marketing data. The existing systems did not scale, and therefore were not prepared to handle the velocity or volume of data expected in the future. Some power users within the organization were executing overwhelmingly complex queries that exceeded system limitations. Finally, the data systems themselves were housed and managed by disparate business units with minimal integration. Mindful of the significant investment the client had made in its Hadoop architecture, Antuit recommended and then implemented a number of changes. The Antuit team helped the client restructure the data lake and created a better data process by partitioning and compressing data, using split-able file formats, and helping them to identify and use the right data types. To create a seamless experience, Antuit was able to leverage multiple test environments to validate approaches and identify ancillary technologies that, once integrated, would keep their data lake running smoothly and efficiently.
运营影响
  • New data lake architecture is fast, reliable, and easily accessible to business decision makers.
  • Antuit established a new set of guidelines for internal users, directing them as to how to retrieve desired data from the lake without bringing the entire system to a halt.
  • Antuit leveraged multiple test environments to validate approaches and identify ancillary technologies that, once integrated, would keep their data lake running smoothly and efficiently.
数量效益
  • Partitioned larger data by effective key.
  • Implemented split-able file format like Sequence, Avro or RC file.
  • Utilized data compress techniques like snappy, bzip2.
  • Used proper data type in hive table.

Case Study missing?

Start adding your own!

Register with your work email and create a new case study profile for your business.

Add New Record

相关案例.

联系我们

欢迎与我们交流!
* Required
* Required
* Required
* Invalid email address
提交此表单,即表示您同意 IoT ONE 可以与您联系并分享洞察和营销信息。
不,谢谢,我不想收到来自 IoT ONE 的任何营销电子邮件。
提交

感谢您的信息!
我们会很快与你取得联系。