IoT GUIDE

By following this list of best practices, your IoT solution can leverage the scalability of Amazon Web Service (AWS) infrastructure and let your team focus on the most important part of your business: your customers.

Whether you’re building a connected solution for asset management, predictive maintenance, yield optimization, or other business goal, the ultimate value returned to your organization will be determined by how well the system meets your domain-specific requirements and individual needs of your customers.

By properly leveraging AWS infrastructure and services as outlined here, the IoT solution you build today will continue to “just work”, no matter how big or how fast your business grows.

BEST PRACTICE #1: BE READY FOR THE DATA TSUNAMI ON DAY 1

It’s going to hit eventually, and whether the flood of incoming data comes from a sudden surge in business, consistent monthly growth, or even a malicious attack, your system should be prepared to handle it right from the start. Fortunately, there are ways to architect your system that allow AWS components to ensure all of your data is always reliably processed. By understanding and adopting proper AWS IoT architecture upfront, you free your team to focus on building application intelligence (where the value is created from the data) and can move ahead confidently with your market strategy no matter how ambitious it may be.

BEST PRACTICE #2: PAY ATTENTION TO THE WEAKEST LINK

AWS IoT Rules allow you to trigger many different actions upon receipt of a message. While all AWS services adapt to different data flow properties, not all services are designed to be used as the single point of entry into the system. Some have behaviors that could prevent all of your data from being processed as desired.

For example, in high volume applications you should consider buffering or queueing incoming data before invoking downstream services like AWS Lambda, so that in the event of subsequent failures the application still has the ability to recover.

With unpredictable spikes in data load, in the time it takes for a Lambda function to spin up there could already be 100’s of thousands of messages in the pipeline. Going to Lambda directly in this example could result in message loss. But If our data is buffered in Kinesis first, this will give downstream systems time to process any large spikes of incoming data.

BEST PRACTICE #3: GET IOT DATA INTO A QUEUE AS SOON AS POSSIBLE

The best way to ensure all incoming data is processed correctly is to go directly from AWS IoT into a Simple Notification Service (SNS) queue which allows AWS fan out your data for processing and ensure the entire flood is reliably captured every time. Many services can subscribe to the SNS topic, including Amazon Simple Queue Service (SQS) and Lambda, which can be called to initiate processing of the queue. By getting your data somewhere ‘safe’ prior to processing – such as a queue, Amazon Kinesis, Amazon S3, and Amazon Redshift – you will not lose any messages due to errors that may occur later due to code or deployment issues.

BEST PRACTICE #4: SOMETIMES THE BEST WAY TO SCALE IS NOT TO SCALE AT ALL

While we’ve shown how your solution can be architected to leverage the scaling power of AWS, sometimes you don’t want to process all of your machine data in the cloud.

One way to reduce traffic in the cloud is to run AWS Greengrass at the edge. Greengrass can intelligently process and filter data locally, eliminating the need to send all device data upstream. You may not need to send 100% of your data all the time. You can capture it all, and hold for a limited time, sending to the cloud only when requested or upon certain error conditions. Alternately, you can choose to send a periodic sample or percentage of device data for use by AWS Machine Learning models and other cloud analytics tools that have been tuned to extract the most value out of even the smallest amounts of data.

While AWS infrastructure allows your system to automatically scale when it needs to, it also provides the ability to process data whenever and however you want to.

BEST PRACTICE #5: CHOOSE THE INGEST THAT IS BEST

Just because you are building an IoT system doesn’t mean all your different types of data have to go through AWS IoT. AWS IoT supports both MQTT and WebSockets, and there are appropriate settings for each service based on recommended use cases. If you ever find yourself bumping up against AWS IoT service limits, there’s a good chance you’re just not using the right service for the particular task.

MQTT is perfect for small payloads of sensor data typical in IoT systems. In a legacy system with log files or XML file dumps, however, it may make more sense to parse the files and send selective values up via MQTT in small chunks. Let’s say you want more than a few values out of the file, it could in fact be more appropriate to send file data via Kinesis or directly to S3 and process it later. MQTT was intended for enabling communications on extremely small embedded devices, why use your device’s limited power for parsing when it can be offloaded to the cloud?

The story is the same for historical and 3rd party data. S3 may be the answer, or it may perhaps be best to ingest into Amazon RDS database or via a your own custom REST API backed by Amazon API Gateway. The key point is that AWS provides several options for data ingest, processing, and storage which should considered as options for different types of data for use in your IoT solution. By taking the step upfront to choose the right AWS tool for the right job, you can maximize the scalability of your overall solution and realize more value from IoT on AWS.

PUTTING IT ALL TOGETHER

Bright Wolf has used these best practices to deliver IoT solutions on AWS across Manufacturing, Oil & Gas, Cold Chain Transportation, Healthcare, Agriculture, Smart Building, Heavy Equipment, and other industries. If you’re looking to get started on a rapid prototype or are facing challenges with an existing system, we can help. Visit us at http://brightwolf.com for more guides and case studies, or to request a demonstration.