Optimizing Autonomous Driving with IoT: A Case Study of Woven Planet and Pachyderm

Woven Planet is a subsidiary of Toyota, dedicated to building the safest mobility in the world. Their focus is on automated driving, and they invest in new technologies, software, and business models that deliver secure, connected, reliable, and sustainable mobility solutions for all. The Automated Mapping team at Woven Planet is responsible for processing petabytes of mapping data to extract semantic information in a fast, iterative, and reproducible manner. They are pioneering new, cutting-edge machine learning solutions to build automotive-grade maps for use in the automated-driving vehicles of today and the autonomous-driving vehicles of tomorrow.

Woven Planet, a subsidiary of Toyota, is focused on building the safest mobility in the world with a particular emphasis on automated driving. The Automated Mapping team at Woven Planet is tasked with creating automotive-grade maps for use in automated and autonomous-driving vehicles. This requires the use of aerial orthographic projection, a method that has been used in the development of consumer-grade navigational maps. However, using this data to meet the rigorous requirements of automated driving at a continental scale is a significant challenge. The maps for automated driving applications need a level of detail, accuracy, and precision far beyond those of their consumer-grade counterparts. This requires processing large volumes of data. The Automated Mapping team needed an orchestration system that could scale to meet elastic workloads, easily toggle between structured and unstructured datasets, and provide long-lived pipeline stability for continuous, region-based map updates.

Woven Planet chose Pachyderm as their orchestration system due to its ability to handle petabytes of machine learning workloads and its compatibility with Kubernetes (K8s), which the Automated Mapping team had strong expertise in. Pachyderm uses the powerful K8s’s orchestration engine to rapidly scale up workers to deal with elastic workloads, making it a suitable solution for the Automated Mapping team’s workflow and pipeline. Woven Planet's Automated Mapping team uses Pachyderm for a range of processes that augment their ingestion-to-training pipeline. They adopt a unique approach to image recognition that divides images into various geolocations and sections. Pachyderm pipelines complement this approach, offering hundreds of pipelines that allow for parallel processing of different road features. This enables the Automated Mapping team to extract semantic information from the data far more efficiently than via a linear approach. The results from these pipelines are collated and passed onto the next stage for new transformations, speeding up time-to-value.

• The use of Pachyderm has significantly improved the efficiency of Woven Planet's Automated Mapping team. With data sourced from various satellite providers and databases, the team has been able to adopt a unique approach to image recognition that divides images into various geolocations and sections. Pachyderm pipelines complement this approach, offering hundreds of pipelines that allow for parallel processing of different road features. This has enabled the team to extract semantic information from the data far more efficiently than via a linear approach. The results from these pipelines are collated and passed onto the next stage for new transformations, speeding up time-to-value and helping Woven Planet deliver maps that have continental-scale coverage and always-on accuracy. Furthermore, the Woven Planet engineers have enjoyed working with the Pachyderm engineering and support teams, highlighting the importance of good support in addition to a powerful platform.

• 200% faster data processing

• Reduced process time by over 50% by only processing changes in the data and taking advantage of Pachyderm's autoscaling features

• Able to scale to petabytes of unstructured map data with Pachyderm's depublication features