Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-20162

Optimize Concurrent Docker Requests for Smart Remote Repository

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: 6.4.3, 6.12.2
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Fresh installations (local or Docker). Tested in mill as well.

      Description

      When concurrently downloading Docker layers, Artifactory will request the same layer multiple times, and will continue to do so until the layer is cached.

      Let's say we have two clusters; cluster A and cluster B.

      B hasn't cached the image yet, and A might be a remote with cached or not.

      Concurrent downloads in cluster B sends out multiple download requests to cluster A for the layers, and only when cluster B completes caching a layer cluster A stops receiving new requests for that layer. 

      For example, an image has 2 layer: layer 1 (1MB), layer 2 (100GB)

      Since layer 1 is small, cluster B caches it quickly, so cluster B only sends out a handful (less than 10 requests out of 100) to cluster A.

      Whereas layer 2 is big, it takes a while to finish caching. Cluster B keeps receiving new requests for layer 2, so it will continue to reach out to cluster A to download layer 2, until the cache completes.

      This results in more than 50 of requests already sent out to cluster A.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              angellom Angello Maggio
            • Votes:
              2 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: