Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-9652

Debian metadata is not being updated in the remote repository in Artifactory and is out of sync when compared to the upstream debian repo

    XMLWordPrintable

    Details

      Description

      Debian metadata files are not being updated in the remote repository in Artifactory and is out of sync when compared to the upstream debian repo. Even when the customer tries downloading the new files the time stamp for the metadata files remains the same in the Artifactory debian repo. The 'Retrieval Cache Period' for the repository in the customer environment is set to 60 seconds.

      We have tried to reproduce this issue on our Artifactory USW online instance, but in our case the debian metadata gets updated when we download a package or the metadata file.

      We were able to see that the download request for a metadata file in the debian cache on the customer's Artifactory instance would show that the resource has expired, which shows that the ‘Retrieval cache period’ has taken effect. But when we try downloading the metadata file from the debian remote repository we see that the file is being downloaded from the cache instead of the upstream debian remote repository.

      When we tested this behavior locally we see that the metadata file is downloaded from the upstream remote once it is expired by the ‘Retrieval cache period’ value.

      Below are the outputs of the cURL requests for the debian metadata files showing the behavior:

      This cURL request is going to the debian remote repo in Artifactory and showing the Last Modified header:

      prince:Downloads joshuah$ curl -I http://artifacts.netflix.com/ubuntu-security-remote/dists/trusty-security/main/binary-amd64/Packages.bz2
      HTTP/1.1 200 OK
      Date: Fri, 25 Mar 2016 01:27:51 GMT
      Content-Type: application/octet-stream
      Content-Length: 447935
      Connection: keep-alive
      Server: Artifactory/4.5.2
      X-Artifactory-Id: netflixdr1b.ntfl-usw2.jfrog.local-netflixdr
      X-Artifactory-Node-Id: NetflixOregon-m
      Last-Modified: Fri, 25 Mar 2016 01:01:00 GMT
      Accept-Ranges: bytes
      X-Artifactory-Filename: Packages.bz2
      Content-Disposition: attachment; filename="Packages.bz2"; filename*=UTF-8''Packages.bz2
      X-Node: nginx1c.ntfl-usw2

      This cURL request is going to the upstream debian repo that the customer is proxying and we can see that the Last Modified timestamp is the same when compared to the previous cURL request which was going to the debian remote in Artifactory:

      prince:Downloads joshuah$ curl -I http://security.ubuntu.com/ubuntu/dists/trusty-security/main/binary-amd64/Packages.bz2
      HTTP/1.1 200 OK
      Date: Fri, 25 Mar 2016 01:28:06 GMT
      Server: Apache/2.2.22 (Ubuntu)
      Last-Modified: Fri, 25 Mar 2016 01:27:00 GMT
      ETag: "6d5bf-52ed575a1c100"
      Accept-Ranges: bytes
      Content-Length: 447935
      Cache-Control: max-age=3233, s-maxage=3300, proxy-revalidate
      Expires: Fri, 25 Mar 2016 02:22:00 GMT
      Content-Type: application/x-bzip2

      The next two cURL request are going to the debian remote repo cache in Artifactory and showing that the Resource is expired:

      prince:Downloads joshuah$ curl -I http://artifacts.netflix.com/ubuntu-security-remote-cache/dists/trusty-security/main/binary-amd64/Packages.bz2
      HTTP/1.1 404 Not Found
      Date: Fri, 25 Mar 2016 01:28:32 GMT
      Content-Type: application/json;charset=ISO-8859-1
      Connection: keep-alive
      Server: Artifactory/4.5.2
      X-Artifactory-Id: netflixdr1b.ntfl-usw2.jfrog.local-netflixdr
      X-Artifactory-Node-Id: NetflixOregon-s
      X-Node: nginx1c.ntfl-usw2

      $ curl http://artifacts.netflix.com/ubuntu-security-remote-cache/dists/trusty-security/main/binary-amd64/Packages.bz2
      {
      "errors" : [

      { "status" : 404, "message" : "Resource has expired" }

      ]
      }

      However, when we try downloading the debian metadata file we see that it is getting the file from the cache and not from the upstream repository which causes the metadata to remain in a stale state even though it is expired.

        Attachments

          Activity

              People

              Assignee:
              yossis Yossi Shaul
              Reporter:
              nihalc@jfrog.com Nihal Reddy Chinna Choudhary (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: