UploadAsync fails when Tags added to blob uploading to Storage Account associated with an IoT Hub

[KiwiBryn]

OS, version, SKU and CPU architecture used: Windows 10 Pro 21H2 , X64

.NET 6.0 v 6.0.202

Device: Desktop (could try on ASUS PE100 Ubuntu, RPI 4B Bullseye if that was helpful)

SDK version used:
"Azure.Storage.Blobs" Version="12.11.0"
"CommandLineParser" Version="2.8.0"
"Microsoft.Azure.Devices.Client" Version="1.40.0"

We have a data capture application (storage account only) which uploads images with "tags" so it is easier to search for images we want to review. Our metrics application (IoT Hub + storage account) does some "inferencing" then uploads telemetry, plus the raw and marked up images to the Storage Account associated with the IoT Hub.

When I tried to add "tags" to an image the upload failed with the error below...

Uploading file TestPayload.txt
Getting SAS URI from IoT Hub to use when uploading the file...
Successfully got SAS URI (https://xxxxxxx.blob.core.windows.net/azureiothubimages/backyard105%2FTestPayload.txt?sv=2018-03-28&sr=b&sig=NZm%2Bk7gaB7PZHDz3ePuwKOxBznEgCNyNeKR4czCsjrs%3D&se=2022-04-18T07%3A14%3A49Z&sp=rw) from IoT Hub
Uploading file TestPayload.txt using the Azure Storage SDK and the retrieved SAS URI for authentication
Failed to upload file to Azure Storage using the Azure Storage SDK due to Azure.RequestFailedException: This request is not authorized to perform this operation using this permission.
RequestId:d1098366-101e-0040-27eb-52831a000000
Time:2022-04-18T06:14:50.4597519Z
Status: 403 (This request is not authorized to perform this operation using this permission.)
ErrorCode: AuthorizationPermissionMismatch

The simplest repro could get is based on [azure-iot-samples-csharp/iot-hub/Samples/device/FileUploadSample

Around line 59 ish

// Works
await blockBlobClient.UploadAsync(fileStreamSource, new BlobUploadOptions());

// Works
BlobUploadOptions blobUploadOptions = new BlobUploadOptions();
await blockBlobClient.UploadAsync(fileStreamSource, blobUploadOptions);

// Works
BlobUploadOptions blobUploadOptions = new BlobUploadOptions()
{
Metadata = new Dictionary<string, string>()
};
blobUploadOptions.Metadata.Add("MetaData1", "1");
blobUploadOptions.Metadata.Add("MetaData2", "2");
blobUploadOptions.Metadata.Add("MetaData3", "3");

// Fails
BlobUploadOptions blobUploadOptions = new BlobUploadOptions()
{
Tags = new Dictionary<string, string>()
};
blobUploadOptions.Tags.Add("Tag1", "1");
blobUploadOptions.Tags.Add("Tag2", "2");
blobUploadOptions.Tags.Add("Tag3", "2");

await blockBlobClient.UploadAsync(fileStreamSource, blobUploadOptions);

Odd thing is adding MetaData works but not Tags

iot_000001.etl.txt

[drwill-ms]

Interesting issue. Obviously, this has to do with the SAS URI provided by IoT Hub and what permissions are on it. I'll check with the IoT Hub service team and possibly the Azure Storage team to find out what's going on here, but it may be slow going. How much of a blocker is this for you?

[drwill-ms]

This appears to be a problem with the SAS URI generated by Azure Storage, or the handling in Storage with a SAS URI with write permissions, not allowing writing to the Tags.

[drwill-ms]

@KiwiBryn What package name and version of the Storage blobs client are you using?

[KiwiBryn]

@drwill-ms copied from project, is this sufficient?

PackageReference Include="Azure.Storage.Blobs" Version="12.11.0"
PackageReference Include="Microsoft.Azure.Devices.Client" Version="1.40.0"
PackageReference Include="Microsoft.Azure.Devices.Provisioning.Client" Version="1.19.1"
PackageReference Include="Microsoft.Azure.Devices.Provisioning.Transport.Amqp" Version="1.16.1"
PackageReference Include="Microsoft.Extensions.Hosting" Version="5.0.0"
PackageReference Include="Microsoft.Extensions.Hosting.Systemd" Version="5.0.1"
PackageReference Include="Microsoft.ML.OnnxRuntime" Version="1.11.0"
PackageReference Include="Yolov5Net" Version="1.0.9"

Is there anything else which would be useful?

[drwill-ms]

I've reached out to a contact on the Azure Storage team to see if they know why we see this error. Hang tight! :)

[KiwiBryn]

Forgot to add it's not a blocker for us.

May cause some issues in about a month as I have a slightly "nasty" work around

[drwill-ms]

@KiwiBryn I've found that the issue lies with an older NuGet package used in Azure IoT Hub service that precedes the introduction of support for allowing the editing of Tags in the SAS URI. Given how work and deployment timelines work, I suspect that'll take months minimum, even if they start today. I've inquired about it with engineering leadership and referred to this case.