# 20.10.4 (Chicago) Release Notes for Cloud
# Resolved Issues
TIP
This update includes mandatory security updates necessary for system operations.
# [86540] - Fixed issue with unsuccessful delete of files
Fixed issue with unsuccessful delete of files, where there is a lot of files in the storage.
When there were too many results in metadata storage, the continuation token was ignored and the file wasn't deleted.
# [86910] - File upload is not transferring all files from IOT devices to long term storage
Resolved the issue when during some dependent services issues (like unavailability of AuthZ or DataAccess) FileUploadProc Azure Function retried the failed operation 5 times and then marked the uploaded blob as poisoned and forget about it. This caused that some client files weren't uploaded because of our infrastructure issues. With the new changes in case of such internal failures, FileUploadProc Azure Function will enter the indefinite regression retry loop until service becomes available again.
Additionally, the auth token caching issue has been fixed.
# Known Issues
# [81239] - Issue with AD token for users with more than 200 group memberships
Known Issue: When adding a user with AD group membership to Admin Portal, users who have more than 200 group memberships (direct & indirect) are not able to login to the portal as the memberships are not being returned in the AD token. This is a limitation on the AD side and needs some major implementation changes to overcome this problem.
Workaround: User can be added under a tenant as a tenant admin to overcome this problem. There is a 3-day time limit on the initial login, after which the user will be deleted automatically if not logged in yet. This is customizable for each client installation,
# [85925] - DCS default of 10 message senders increases response time six times
According to recent findings, Message Sender works faster when having 2 instances instead of more of them. That's why as a temporary workaround it's recommended to change PlatformMessageSender instances to 2, instead of the default 12.
# [87636] - DA Services API returning 500 errors
Max number of concurrent requests in TSI gen1 is equal to 10. This is an external (Microsoft) limitation.
# [81401] - Data Ingress limit limitation for Time Series Insights
Three levels of Time Series Insights are supported, S1, S2, and P1.
S1 supports up to 120 1K TSI events per second and S2 supports 1200 1K TSI evens per second. If the TSI event is greater than 1K in size, maximum ingress will be reduced. For example, if all events are between 3K and 4K in size, the maximum ingress is one forth the maximum supported, 30 TSI events per second for S1 and 300 TSI Events for S2.
P1 is based on a bandwidth of data stored. TSI P1 supports 6MB/s of ingress. However, other components restrict ingress to 1MB/s at this time. 1MB/s is approximately 6000/s of basic telemetry.
# [35242] - DA-groupBy field in the Data Access query doesn't support "int"
In TSI gen1 queries grouping can be done only on columns that are denoted with the data type. The only known columns and their data types are objectId, model, timestamp, and variable/event/alarm. Since the "value" column can be any simple data type (string, double, Boolean, timestamp) or complex object, the system cannot determine at query time what to group by for data type. By default, the grouping operation data type is assumed to be "string".
# [45735] - QEL expressions with with sentence: "name in ['x']" doesn't work
MongoDB MongoCollection cannot handle certain array arguments. Am application may see the following: creating a condition with IN ['x'] will return 400 Bad Request.
Creating a combination of "name = 'x' OR "name = 'y' " should be used instead as a workaround. For additional information, please reference the following article, MongoDB site, issue https://jira.mongodb.org/browse/CSHARP-2727.
# [55864] - Solution create audit log shows wrong event for audit log
The audit log for creating a solution is shown incorrectly as an update instead of create.
# [59246] - Requests for bearer token for new application can return to Bad Request
Known Issue: When concurrent requests to get a bearer token are sent, a client can receive a Bad Request response.
Workaround: The recommendation from Microsoft is the application must wait for a few seconds before trying to get the token for the application that has been created. According to Microsoft, it takes a maximum of 60 seconds to replicate the Application Settings across Azure regions.
# [62908] - Principle Manager API fails to remove tenants - BadGateway
The problem can occur based on concurrent requests to the principal manager API. The Principal Manager APIs are using Azure B2C services to create Applications for business entities, e.g. Application, Solution, etc. The workflow in the PM is sequential and dependent on the result of the B2C operation. After a successful result from the B2C operation, the request is further processed to provide the respective response to the caller.
For any B2C-related request, some buffer time needs to be provided so that the action can be completed.
It is advised to maintain a gap of 60 secs between two requests.
# [64217] - 'Service not available' error when creating App (MS ticket: 2103190050000136)
Due to a bug in the B2C call from Principal Manager, the Application name cannot contain the '&' character (ServiceUnavailable error is thrown). https://graph.microsoft.com/v1.0/applications endpoint fails when ampersand is present in displayName in the body. The ticket to Microsoft has been raised and is awaiting resolution.
# [68309] - Unable to search for a file using user token after upload
Know Issue: When searching for files uploaded via Edge, requests using a user token are failing when the number of objects exceeds 500.
Workaround: When querying, the objectid, along with the path, can be passed in QEL format to overcome this limitation.
# [73073] - Bad Request on variable subscription for long request that do not include objectIDs
Known Issue: When creating a subscription with a user token, filters that do not contain an objectId may result in an error preventing the subscription from being created. The likely HTTP response code may be 4xx. This is due to a limitation of a service bus filter only allowing for up to 1024 characters. This issue is often observed with a large ability-condition header that cannot be broken down into small enough filters for service bus.
Workaround: To workaround, include an objectId in the filter property for the Data Access request. Up to 40 objectids can be included in the filter.
Background tokens do not have tenancy therefore the authorization service does not optimize the ability-condition based on the given objectId in the filter.
# [73453] - A Solution cannot register more than 40K devices
The workaround for this issue is to create a duplicate solution and add additional devices if you exceed the limit of 40K devices.
The limitation is caused by a maximum 2MB document size in CosmosDb.
# [73963] - A Latency issue can cause new application created by principal manager to be constructed without secrets
Known Issue: Occasionally a newly created application using principal manager service will be created without secrets causing the app to become unusable because a bearer token cannot be obtained.
Workaround: If this is observed, create the application again after about waiting 60 secs.
# [74595] - User cannot access applications when the "read" permission is limited to "user" delegation
Known Issue:
Query apps endpoint - passing 'user' instead of 'User' for delegation parameter will return empty results.
Get apps endpoint - passing 'User' instead of 'user' for delegation parameter returns empty results.
Workaround: When querying for applications using the "Query apps" or "Get apps" endpoint, limited to user delegation, pass (delegation='user' OR delegation='User') for delegation parameter to get the expected results
# [75339] - Sorting functionality which has been implemented as part of Pagination & Searching feature in Principal Manager APIs, is case sensitive
Sorting functionality which has been implemented as part of the Pagination & Searching feature in Principal Manager APIs is case sensitive.
For example: when trying to sort a set of tenants, {ABB01, Robotics01, abb02, Volvo01, robotics02, volvo02} the result which will be returned when sorting ascending is {ABB01, Robotics01, Volvo01, abb02, robotics02, volvo02}
# [76007] - DSL query escape sequence handling for backward slash() in property value filter is not consistent
Known Issue: When using the backslashes ("") in the object model properties and then trying to query them using DSL, the user cannot obtain it by a single escape character ("\"), which is expected behavior.
Workaround: The workaround is to use double escaping in the DSL query ("\\").
For example, having property:
{
"browseName": {
"value": "some\\path"
}
}
one needs to use the DSL:
models(...).hasProperty("browseName", "some\\\\\\\\path")
# [77345] - When creating a solution or resource, principle manager service sporadically returns a 400 Bad Gateway response code
Known Issue: When creating a solution or a resource, Microsoft Graph API occasionally will return a 400 Bad Gateway response code with the message, "One or more of your reply URLs is not valid". As a result, the Solution is not created.
Workaround: The end user will need to resubmit the request
# [77522] - AuditLog events Count Mismatch for Device Created, Updated and Deleted operations
The body of platform events is stored in Audit Logging storage. However, in some cases, this body contains a JSON object which exceeds Azure Table Storage column limitations.
In this case, when a platform event body is longer than 16k characters, Audit Logging saves the following warning
information into the "data" column: {"auditLogInformation": "Event body too long"}.
The original body of the event is not saved, however, the user can still navigate to actual changes by using the event correlationId.
Limitations source:
size of an entity in table storage up to 1MB (https://docs.microsoft.com/en-us/azure/cosmos-db/table-storage-design-guide#capacity-considerations)
size of single string column up to 64 Kb (which equals 32k characters, https://docs.microsoft.com/en-us/rest/api/storageservices/understanding-the-table-service-data-model#property-types)
# [79098] - TSI storage doubled (costs) and max throughput decreased when DPP status code processing is enabled
Data quality decoration increases the total size of the telemetry message significantly.
An original message may look like this:
{
"objectId": "2B129E4C-0944-4534-8E8B-DEB49D8AF0AC",
"model": "abb.somedomain.somemodel",
"variable": "SomeVariableName",
"timestamp": "2018-05-217T23:00:00Z",
"value": 42,
"quality" : 1073741954
}
After quality decoding it may look like this:
{
"objectId": "2B129E4C-0944-4534-8E8B-DEB49D8AF0AC",
"model": "abb.somedomain.somemodel",
"variable": "SomeVariableName",
"timestamp": "2018-05-217T23:00:00Z",
"value": 42,
"quality" : 1073741954,
"qualityFlags" : {
"validity" : "uncertain",
"limit" : "low",
"historian" : "interpolated"
}
}
This in turn has a direct impact on:
- the total capacity of data that Ability Platform ingress pipeline may accept (Azure Event Hubs limits that to 20MB/sec.)
- the total amount of data being stored to TSI, which has a direct impact on the cost of the system
Data quality decoration is turned off by default and not recommended for use at this time.
A future update is planned to provide better control over the extra space used when this feature is enabled.
# [80407] - Object cannot be created due to limit of 64kB for create object gremlin query
Due to the CosmosDB gremlin API limitation it is not possible to create an object model with a size bigger than 64KB (exactly 63035B).
The request with too big object model returns 400 (BadRequest) with detailed error message:
{
"errors":{
"code": 400,
"title": "BadRequest",
"detail": "Object model query exceeded maximum length. Allowed query size: 65400. Consider decreasing length of string values assigned in object model.",
"id": "<correlationId>",
"status": "BadRequest"
}
}