[RTFACT-14327] Windows images referencing foreign layers are not cached inside artifactory in version 5.3 Created: 02/Jun/17  Updated: 14/Nov/18  Resolved: 16/Sep/18

Status: Resolved
Project: Artifactory Binary Repository
Component/s: Docker
Affects Version/s: 5.10.4
Fix Version/s: 6.4.0

Type: New Feature Priority: Normal
Reporter: Spencer Flanders Assignee: Rotem Kfir
Resolution: Fixed Votes: 8
Labels: None

Issue Links:
Duplicate
is duplicated by RTFACT-15237 artifactory do not pulling some layer... Resolved
is duplicated by RTFACT-13394 Add support to rewrite foreign layers... Resolved
Relationship
relates to RTFACT-10305 Add support for manifest with referen... Resolved
Assigned QA: Matan Katz

 Description   

We were testing docker updates for 5.3, hoping to see fixes for remote repo caching regarding docker images for windows, which our rep Ben Irizarry thought was there. We want to force all our dependency management to go through this tool so we can further lock down our firewall. From our analysis we found this:

  • Remote caching is only storing the manifest and a single sha256 file containing xml
  • Client appears to be handling downloading the image pieces, so firewall has to allow access to those locations
  • At this point a docker client will need access to: docker.io, go.microsoft.com, az896309.vo.msecnd.net
  • Testing blocking links on my local firewall and was able to verify that the client is initiating pulling those secondary links
  • Artifactory only stores images for the custom stuff we build

Example manifest (microsoft/nanoserver/10.0.14393.953)

{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
    "config": {
        "mediaType": "application/vnd.docker.container.image.v1+json",
        "size": 582,
        "digest": "sha256:18a0d32a4b98e8a3e1ab7eb33b2be75b4826cbf43754961db08101b52bce0840"
    },
    "layers": [{
            "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
            "size": 252691002,
            "digest": "sha256:bce2fbc256ea437a87dadac2f69aabd25bed4f56255549090056c1131fad0277",
            "urls": [
                "https://go.microsoft.com/fwlink/?linkid=837858"
            ]
        },
        {
            "mediaType": "application/vnd.docker.image.rootfs.foreign.diff.tar.gzip",
            "size": 114925341,
            "digest": "sha256:58f68fa0ceda734a980c12dedf782342f892e218bba3c74ded58bfabed652ba1",
            "urls": [
                "https://go.microsoft.com/fwlink/?linkid=844835"
            ]
        }
    ]
}

sha256__18a0d32a4b98e8a3e1ab7eb33b2be75b4826cbf43754961db08101b52bce0840

{  
   "config":{  
      "Hostname":"",
      "Domainname":"",
      "User":"",
      "AttachStdin":false,
      "AttachStdout":false,
      "AttachStderr":false,
      "Tty":false,
      "OpenStdin":false,
      "StdinOnce":false,
      "Env":null,
      "Cmd":[  
         "c:\\windows\\system32\\cmd.exe"
      ],
      "Image":"",
      "Volumes":null,
      "WorkingDir":"",
      "Entrypoint":null,
      "OnBuild":null,
      "Labels":null
   },
   "created":"2017-03-08T08:52:53.2940846-08:00",
   "os":"windows",
   "os.version":"10.0.14393.953",
   "rootfs":{  
      "type":"layers",
      "diff_ids":[  
         "sha256:6c357baed9f5177e8c8fd1fa35b39266f329535ec8801385134790eb08d8787d",
         "sha256:75c873eec901cd63e8989874fe2a8056354876ffd7e877d3e3216d3edbf70f94"
      ]
   }
}
C:\Code\Corp\POC\Containers\Network\Linux>docker version
Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Tue Mar 28 00:40:02 2017
 OS/Arch:      windows/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.24)
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Tue Mar 28 00:40:02 2017
 OS/Arch:      windows/amd64
 Experimental: true

Artifactory version: 5.3.0 rev 50045

Is this expected behavior and if so, are there any plans to support caching these items for windows?



 Comments   
Comment by Morgan Howard [ 11/Oct/17 ]

We are seeing this as well Artifactory 5.4.6. Any updates? Docker version is below:

>docker version
Client:
 Version:      17.09.0-ce
 API version:  1.32
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:40:09 2017
 OS/Arch:      windows/amd64

Server:
 Version:      17.09.0-ce
 API version:  1.32 (minimum version 1.24)
 Go version:   go1.8.3
 Git commit:   afdb6d4
 Built:        Tue Sep 26 22:50:27 2017
 OS/Arch:      windows/amd64
 Experimental: true
Comment by Torsten Liermann [ 22/Mar/18 ]

Please handle this as a bug fix, not as a feature. Thanks.

Comment by Kurt Bomya [ 06/Aug/18 ]

Like Torsten, I'd also like to see this treated as a bug, not a feature!

Comment by Alex Dvorkin [ 20/Sep/18 ]

Moved back to QA, since we are reverting fat manifests. Therefore, this feature should be re-tested for side effects.

Comment by Alex Dvorkin [ 25/Sep/18 ]

Verified on 6.4.0-m006

Generated at Sat Aug 24 22:03:38 UTC 2019 using JIRA 7.6.3#76005-sha1:8a4e38d34af948780dbf52044e7aafb13a7cae58.