What is Azure Blob storage on IoT edge?

It is a lightweight Azure consistent module that provides local block blob storage. A blob storage module on your IoT Edge device behaves like an Azure blob service, except the blobs are stored locally on your IoT Edge device. You can access your blobs using the same Azure storage SDK methods or blob API calls that you’re already used to.

Abilities and features:

• Automatic uploading the data on the local blob storage account to Azure and provides support for poor internet connectivity.
• Automatically delete the data from the local storage account.

Common scenarios:

Following are the common scenarios where the module uses block-level uploads:

• When you want to store the event-based recording of video feed from a parking lot of only those timestamps when a motion is detected, so you can keep it locally until it can be transferred or processed to the cloud. It updates some blocks of a previously uploaded block blob or appends new blocks to an append blob. It uploads only the updated blocks and not the whole blob.
• When devices are in a place with limited connectivity, consider a scenario where you want to collect data from an offshore ship that sends oil reserves data in the sea. It will upload blob when unexpected process termination like power failure or network failure happens during a blob upload. When the connectivity is back again, it uploads only the remaining blocks and not the whole blob.
• When you want to set up a motion detection system at home or office where you efficiently process the data locally to get low latency access to the data, you can respond to emergencies as quickly as possible.
• Rather than storing entire video recordings, we can store data that is recorded event-based. This will reduce bandwidth costs and avoid transferring terabytes of data to the cloud. You can process the data locally and send only the processed data to the cloud.

Configuration of module

• Set up IoT Edge device running Azure IoT edge runtime in VS code
• Add the following module in template file and deploy it

A. Module nameàlocalstorage

B. Link to connect local storage account with account key

C. Storage path to store container data

Scenario 1: Upload data from device to cloud

This data can be photos, videos, text, finance data, hospital data, or any other unstructured data as required. The above code snippet can be used in various use cases like you have collected your data (any form of data images, video etc) in your local storage account and now you want to push them from device to cloud using Azure storage explorer.

Following is the explanation for above code snippet:

1. 1. Module name –> localstorage

Connection string to your local storage account

1. 2. Order of upload is set to “OldestFirst” (or “NewestFirst” according to requirement) this means our oldest data will be uploaded first. Followed by connection string to Azure storage account where the data will be uploaded.

Specify source and target container names in storageContainerForUpload properties.

1. 3. Here, source –> cont1
      target –> edge-cont1 in azure blob storage (ABS)
   4. Here, source –> cont2
      target –> template string to specify target container in ABS
   5. Here, source –> cont3
      target –> automatically assign target
   6. After data upload, “deleteAfterUpload” property will automatically delete the data from local storage account.

Scenario 2: Apply data retention policies

This scenario is a continuation of the previous one where we apply data retention policies as well. For example, you have data in your local storage account, and complete the migration to cloud. Now, these policies will work on data after on-cloud migration.

7. “deviceAutoDeleteProperties” defines how long the data needs to be kept and when to delete it when it is no longer needed. Set it true to turn on this feature.

8. Module will delete the data from local blob storage account after 5 minutes of its last modified time. “retainWhileUploading” will retain the blob while it is uploading even after “deleteAfterMinutes” value exceeds. These properties can be applied in many combinations to explore data retention policies in action.

Download and install Azure storage-explorer and connect to the local storage account. Create the source containers and explore the above scenarios.

Supported functionalities

deviceToCloudUpload functionality

deviceAutoDelete functionality

deviceToCloudUpload functionality in disconnected scenario

Block diagram

Supported storage operations:

Blob storage modules on IoT Edge use the Azure Storage SDKs and are consistent with the 2017-04-17 version of the Azure Storage API for block blob endpoints.

Because Azure Blob Storage supports not all Azure Blob Storage operations on IoT Edge, here are the supported and unsupported operations list:

 

External references:

• How to store data blob?
• Deploy blob storage on module to your device – Azure IoT Edge | Microsoft Docs
• https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-portal

• How to use Azure Blob Storage on IoT Edge