Edge Device to Cloud (D2C) Message Compression

Introduction

This feature allows to decrease the number of bytes sent over the wire, which is most useful when:

• your device operates in an area where Internet connection is not "cheap"
• your platform instance is configured on a lower SKU
• your messages may exceed the IoT Hub message limit (256 kB) in the device to cloud direction

The following V2 device actions support compression:

• Models (model.create, model.update, model.patch, model.delete, model.query)
• Types (type.create, type.delete, type.query)
• References (reference.create, reference.delete)
• Extensions (extension.create, extension.get)
• Identity (identity.create, identity.update, identity.delete)
• Batching (batch.execute)

Configuring Compression for Device Actions

The compression can be enabled on the edge by adding the below configuration in the edge configuration type definition:

properties ► systemModules ► edge-proxy ► configuration ► deviceAPI ► contentEncoding ►

"deviceAPI": {
    "contentEncoding": {
        "description": "Compression format to be used while sending device to cloud actions. Setting the value to 'Off' represents 'No Compression'",
        "dataType": "string",
        "enum": [
            "gzip",
            "deflate",
            "off"
        ],
        "isMandatory": false
    }
}

• This feature is only available with apiVersion V2. To understand how to enable apiVersionV2, refer to the section Enabling Module API V2 in your Edge.

• The compression formats supported by this feature are gzip and deflate.

• The maximum size of the device message that can be sent from an edge device can be controlled using the property below:

"maxDeviceMessageSize": {
    "description": "Maximum device message size, in kb",
    "dataType": "integer",
    "isMandatory": true,
    "value": 256,
    "minimum": 2,
    "maximum": 256
}

This property also controls the maximum size of the batched telemetry that can be transmitted from the edge device.

• The contentEncoding property in the edge configuration model only controls the compression of device action messages. The telemetry compression is still controlled by the existing property in the edge configuration available in: properties ► telemetry ► compression ►.

"telemetry": {
    "compression": {
        "description": "Compression format to be used while sending telemetries. Setting the value to 'Off' represents 'No Compression'",
        "dataType": "string",
        "enum": [
            "gzip",
            "deflate",
            "off"
        ]
    }
}

• The default value for the property deviceAPI ► contentEncoding as well as telemetry ► compression is utf-8 which is equivalent to the current behavior when no compression is enabled.

• Setting the contentEncoding value to any other value than gzip or deflate, for example to off, defaults the encoding format to utf-8.

• The IoT Hub message limit in the cloud to device direction is 64KB, and the limit in the device to cloud direction is 256KB.

Exception Handling

1. When an uncompressed message sent by the edge module is too large to be compressed below the configured size defined in maxDeviceMessageSize, the Edge will send a negative ack to the edge module on the respective acknowledgment topic defined for that device action.

   {
     "success": false,
     "code": "request_too_large",
     "details": "Request message size is too large and cannot be sent to IoT Hub",
     "excess": <Request message body excess size>,
     "size": <Request message body size uncompressed>,
     "compressed_size": <Request message body size compressed>
   }
   

   Header properties:

   Key	Value
   correlationId	<reused from request>
   target	<reused from request>

   Body (JSON format):

   Key	Mandatory/Optional	Value Type	Allowed Values	Description
   success	Mandatory	String	false	Boolean value indicating if the request was successful or not
   code	Mandatory	String	request_too_large	Error code indicating the type of failure
   excess	Mandatory	Number	Floating point number above 0	Request message body excess size above IoT Hub limit, in kB. If the message body is uncompressed, this property contains uncompressed excess size. If the message body is compressed, this property contains compressed excess size
   size	Mandatory	Number	Floating point number above 0	Request message body size uncompressed, in kB
   compressed_size	Optional	Number	Floating point number above 0	Request message body size compressed, in kB. Present only when message body is compressed

2. When a compressed cloud to device message (C2D) is received by the Edge and decompression fails, the edge will send a negative ack to the edge module on the respective acknowledgement topic defined for that device action.

   {
     "success": false,
     "code": "decompression_error",
     "details": "Response message cannot be decompressed by Edge Proxy"
   }
   

   Header properties:

   Key	Value
   correlationId	<reused from request>
   target	<reused from request>

   Body (JSON format):

   Key	Mandatory/Optional	Value Type	Allowed Values	Description
   success	Mandatory	String	false	Boolean value indicating if the request was successful or not
   code	Mandatory	String	decompression_error	Error code indicating the type of failure
   details	Optional	String		A message indicating the reason for the error