Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-20472

Can't upload files larger than 12gb to Azure blob storage via Artifactory

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Affects Version/s: 6.13.1
    • Fix Version/s: 6.16.0
    • Component/s: Azure
    • Labels:
      None
    • Severity:
      High

      Description

      Steps to reproduce:
      1. Start Artifactory with Azure blob storage fs
      https://www.jfrog.com/confluence/display/RTF/Configuring+the+Filestore#ConfiguringtheFilestore-AzureBlobStorageBinaryProvider
      2. Upload file larger than 12gb

      After a while, when the eventual fs will try to upload the file to Azure, you will see this error:

      2019-10-30 13:58:02,505 [pool-8-thread-5] [WARN ] (o.j.s.b.p.RetryBinaryProvider:157) - Failed to add blob to next binary provider for the '1' time, a retry will start in seconds
      2019-10-30 13:58:07,656 [pool-8-thread-5] [WARN ] (o.j.s.b.p.RetryBinaryProvider:157) - Failed to add blob to next binary provider for the '2' time, a retry will start in seconds
      2019-10-30 13:58:12,790 [pool-8-thread-5] [WARN ] (o.j.s.b.p.RetryBinaryProvider:157) - Failed to add blob to next binary provider for the '3' time, a retry will start in seconds
      2019-10-30 13:58:17,956 [pool-8-thread-5] [WARN ] (o.j.s.b.p.RetryBinaryProvider:157) - Failed to add blob to next binary provider for the '4' time, a retry will start in seconds
      2019-10-30 13:58:23,094 [pool-8-thread-5] [WARN ] (o.j.s.b.p.RetryBinaryProvider:157) - Failed to add blob to next binary provider for the '5' time, a retry will start in seconds
      2019-10-30 13:58:28,265 [pool-8-thread-5] [ERROR] (o.j.s.b.p.RetryBinaryProvider:164) - Failed to add blob to next binary provider
      java.io.IOException: Failed to put blob 'a44af2302a6a5871104e48800b091b6eec3e8735' to Azure blob storage
      	at org.artifactory.addon.filestore.type.azureblobstorage.AzureBlobStorageBinaryProvider.addBigStream(AzureBlobStorageBinaryProvider.java:271)
      	at org.artifactory.addon.filestore.type.azureblobstorage.AzureBlobStorageBinaryProvider.addStream(AzureBlobStorageBinaryProvider.java:200)
      	at org.jfrog.storage.binstore.providers.RetryBinaryProvider.addStream(RetryBinaryProvider.java:154)
      	at org.jfrog.storage.binstore.providers.RetryBinaryProvider.addStream(RetryBinaryProvider.java:162)
      	at org.jfrog.storage.binstore.providers.RetryBinaryProvider.addStream(RetryBinaryProvider.java:162)
      	at org.jfrog.storage.binstore.providers.RetryBinaryProvider.addStream(RetryBinaryProvider.java:162)
      	at org.jfrog.storage.binstore.providers.RetryBinaryProvider.addStream(RetryBinaryProvider.java:162)
      	at org.jfrog.storage.binstore.providers.RetryBinaryProvider.addStream(RetryBinaryProvider.java:162)
      	at org.jfrog.storage.binstore.providers.RetryBinaryProvider.addStream(RetryBinaryProvider.java:69)
      	at org.artifactory.addon.filestore.eventual.task.EventuallyPersistedAddFileTask.run(EventuallyPersistedAddFileTask.java:71)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.RuntimeException: Failure occurred during blob upload
      	at org.artifactory.addon.filestore.type.azureblobstorage.AzureBlobStorageBinaryProvider.doBlockUpload(AzureBlobStorageBinaryProvider.java:322)
      	at org.artifactory.addon.filestore.type.azureblobstorage.AzureBlobStorageBinaryProvider.listFileBlocksTobeUploaded(AzureBlobStorageBinaryProvider.java:309)
      	at org.artifactory.addon.filestore.type.azureblobstorage.AzureBlobStorageBinaryProvider.addBigStream(AzureBlobStorageBinaryProvider.java:262)
      	... 14 common frames omitted
      Caused by: com.microsoft.azure.storage.StorageException: The request body is too large and exceeds the maximum permissible limit.
      	at com.microsoft.azure.storage.StorageException.translateException(StorageException.java:89)
      	at com.microsoft.azure.storage.core.StorageRequest.materializeException(StorageRequest.java:305)
      	at com.microsoft.azure.storage.core.ExecutionEngine.executeWithRetry(ExecutionEngine.java:175)
      	at com.microsoft.azure.storage.blob.CloudBlockBlob.uploadBlockInternal(CloudBlockBlob.java:904)
      	at com.microsoft.azure.storage.blob.CloudBlockBlob.uploadBlock(CloudBlockBlob.java:876)
      	at org.artifactory.addon.filestore.type.azureblobstorage.AzureBlobStorageBinaryProvider.doBlockUpload(AzureBlobStorageBinaryProvider.java:319)
      	... 16 common frames omitted
      

      The file will be stuck in the eventual and will never get to the Azure blob storage

      Uploading a 12gb works - it gets to the Azure blob storage

      Note, increasing the batch size by adding this did not help:

      <multipartElementSize>4194304</multipartElementSize>
      

        Attachments

          Issue Links

            Activity

                People

                Assignee:
                tamirh Tamir Hadad
                Reporter:
                avivb Aviv Blonder
                Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    Sync Status

                    Connection: RTFACT Sync
                    RTMID-20472 -
                    SYNCHRONIZED
                    • Last Sync Date: