Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-26582

Tar archive download incomplete with files >8GB

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Also occurs in 

      Artifactory  7.29.7. single node, On-prem, Derby

       

    • Severity:
      High
    • Location:
      External

      Description

      When repository folder contains a file bigger than 8GB, attempting to download a folder in tar or tar.gz results in an incomplete archive. The archive is not corrupted, passes validation, but includes only files up to the one that is bigger than 8GB.

      Steps to reproduce:
      1. Upload any two regular files (<8GB) into a repository.
      2. Upload a file >8GB with name between the other two.

      dd if=/dev/urandom bs=1M count=8200 | curl -n -T - https://artifactory-sbox.local/artifactory/test-generic-local/garbage
      

      3. Inspect repository contents. Make sure that >8GB is between other two.

      $ curl -n https://artifactory-sbox.local/artifactory/api/storage/test-generic-local?list
      {
        "uri" : "https://artifactory-sbox.local/artifactory/api/storage/test-generic-local",
        "created" : "2021-12-06T14:35:11.673Z",
        "files" : [ {
          "uri" : "/foo",
          "size" : 0,
          "lastModified" : "2021-10-14T08:29:07.361Z",
          "folder" : false,
          "sha1" : "da39a3ee5e6b4b0d3255bfef95601890afd80709",
          "sha2" : "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
        }, {
          "uri" : "/garbage",
          "size" : 8598323200,
          "lastModified" : "2021-12-06T13:55:47.148Z",
          "folder" : false,
          "sha1" : "3d36585cbd4c6bdfc665908c5696aa4fe4a4c10f",
          "sha2" : "04f31bc32b019168112ef6a113dfac61316c51573718695281cb0384412cd3ae"
        }, {
          "uri" : "/sesion 6.pdf",
          "size" : 29013432,
          "lastModified" : "2021-10-14T08:29:31.906Z",
          "folder" : false,
          "sha1" : "3d5eef7f3e8964c3b0ea5842d0cd111c1fa68c4f",
          "sha2" : "577b769e9948e92196503ecb6b47c854977444be148da21d4f6e9bbc8ef49c55"
        } ]
      }
      

      4. Download repo contents in tar format. Observe response code 200.

      $ curl -nvs -o archive.tar https://artifactory-sbox.local/artifactory/api/archive/download/test-generic-local/?archiveType=tar
      ...
      > GET /artifactory/api/archive/download/test-generic-local/?archiveType=tar HTTP/2
      > Host: artifactory-sbox.local
      > authorization: Basic [redacted]
      > user-agent: curl/7.68.0
      > accept: */*
      >
      
      < HTTP/2 200
      < date: Mon, 06 Dec 2021 14:44:47 GMT
      < content-type: */*
      < server: Artifactory/6.21.0
      < x-artifactory-id: 30ccddcbc7b8b3684f3c5375b025de377d41f7e5
      < x-artifactory-node-id: artifactory-sbox.local
      <
      { [1536 bytes data]
      

      5. Test result archive. Note that validation passes, but only files before >8GB are present; neither garbage nor sesion 6.pdf got into the archive.

      $ tar tvf archive.tar
      -rw-r--r-- 0/0               0 2021-12-06 15:44 foo
      

      Since Artifactory uses Apache Commons Compress to handle tar archives, it should initialize TarArchiveOutputStream with appropriate big number mode, otherwise the library will fail on a >8GB file. This error however only gets a non-meaningful notice in artifactory.log:

      2021-12-06 14:44:27,976 [http-nio-8081-exec-8] [ERROR] (o.a.d.FolderArchiveStreamer:111) - Error executing folder download: No current entry to close
      

      and the client is served without errors, but with incorrect data.

        Attachments

          Activity

              People

              Assignee:
              Unassigned
              Reporter:
              akm022 Krzysztof Malinowski
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:

                  Sync Status

                  Connection: RTFACT Sync
                  RTMID-26592 -
                  SYNCHRONIZED
                  • Last Sync Date: