Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-18259

Storage info API should not account duplicate docker layers

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None

      Description

      We see this in /artifactory/api/storageinfo

       { 
        repoKey: "bi", 
        repoType: "LOCAL", 
        foldersCount: 535, 
        filesCount: 3178, 
        usedSpace: "412.72 GB", 
        itemsCount: 3713, 
        packageType: "Docker", 
        percentage: "0.23%" 
       } 

      However, we have only uploaded ~1900 files, totalling ~156Gb.  The difference 
      is in the way client and server are handling reused docker image layers. 

      We are following good practice and automatically building new images as part 
      of our CI pipeline where the top layer is generally the only changing layer.   
        When pushing, most of the layers are not pushed as artifactory returns 
      'Layer already exists'. 

      The artifactory doc also implies shallow copy/links are used to handle 
      knowing when orphaned layers can be deleted. 

      Example - two consecutive, automated image publishes: 

      /artifactory/bi/bi/contentstorage/1.1.0.0.0-20180806060046/ 
      /artifactory/bi/bi/contentstorage/1.1.0.0.0-20180806150421/ 

      4 layers are common, ~2.7Gb.  1 layer changes, 10Kb each time. 

      Docker push/artifactory knows this, as it reports `Layer already exists¿. 

      The storage info should be ~2.7Gb, not ~5.4Gb and the storageinfo REST call 
      should reflect the activiyy from the client's perspective including docker 
      semantics. 

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              daniel.f.fernandez Daniel Fernandez
            • Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: