Problem creating datasets with Azure storage when multi file about clearml

[jkhenning]

Hi @SamGalanakis , this seems perhaps to be related to the azure python package version - can you share the python packages versions you're using?

[SamGalanakis]

Hi @jkhenning this is the pip freeze

requirements.txt

[SamGalanakis]

Also I see that it does store the main data but fails on some metadata / auxillary files.

RequestId:48002e24-301e-005a-022e-c20c19000000
Time:2024-06-19T09:52:41.5150691Z</Message><AuthenticationErrorDetail>Authentication scheme Bearer is not supported in this version.</AuthenticationErrorDetail></Error>)
2024-06-19 09:52:43,794 - clearml.metrics - ERROR - Not uploading 1/5 events because the data upload failed
Uploading dataset changes (8 files compressed to 162.15 MiB) to azure://clearmltest.blob.core.windows.net/clearml
File compression and upload completed: total size 162.15 MiB, 1 chunk(s) stored (average size 162.15 MiB)

[SamGalanakis]

@jkhenning Any update on this?

[jkhenning]

Hi @SamGalanakis,

I've looked around for this error, it seems related to either a misaligned system clock, or to some headers missing from the request (even though I found no change in documentation regarding this).

Since there was no chance in the ClearML SDK code handling this process, I would first try to downgrade the azure-blob-storage version (perhaps try 12.0.0) and see if it helps