4 - Normal SENSITIVE DATA HAS BEEN REMOVED FROM THIS TICKET ACCORDING TO JFROG'S INTERNAL POLICY.
Symptoms: The "apt-get" client returns a checksum mismatch when downloading packages from certain remote repositories. For example, http://mirror.corbina.net/debian/ uses this metadata file.
Steps to reproduce:
- Set up a local Debian repository, and deploy 1 .deb package to it
- Create an empty SHA256SUMS file and upload to: /dists/xenial/main/binary-x86_64/current/images/SHA256SUMS
- On a second Artifactory, set up a remote repository going to the Debian Local
- Download the Debian package, the metadata (Release + Packages), and the SHA256SUMS file
This caches the metadata - Upload a second Debian package to the Debian-local, which updates the metadata
- Upload an updated SHA256SUMS file to replicate updated metadata
- Zap the Debian-remote repository cache
- Re-download the Packages and SHA256SUMS files, observe that the Packages file is updated but the SHA256SUMS file is not
A ?trace call against both reveals that metadata that has expired is updated, but the SHA256SUMS file isn't counted as metadata:
[Packages]
Request ID: 10c98880
Repo Path ID: debian-remote-test:dists/xenial/main/binary-x86_64/Packages
Method Name: GET
User: admin
Time: 2019-01-31T23:48:37.998Z
Thread: http-nio-8081-exec-9
Steps:
2019-01-31T23:48:37.998Z Received request
2019-01-31T23:48:37.999Z Request source = RESTRICTED_IP, Last modified = 31-12-69 23:59:59 +00:00, If modified since = -1, Thread name = http-nio-8081-exec-9
2019-01-31T23:48:37.999Z Executing any BeforeDownloadRequest user plugins that may exist
2019-01-31T23:48:37.999Z Retrieving info from {} repository '{}' type
2019-01-31T23:48:38.001Z Found the resource in the cache - checking for expiry
2019-01-31T23:48:38.001Z Returning resource as expired
2019-01-31T23:48:38.001Z Executing any AltRemotePath user plugins that may exist
2019-01-31T23:48:38.002Z Appending matrix params to remote request URL
2019-01-31T23:48:38.002Z Using remote request URL - http://10.138.0.2:8081/artifactory/debian-local/dists/xenial/main/binary-x86_64/Packages
2019-01-31T23:48:38.002Z Executing HEAD request to http://10.138.0.2:8081/artifactory/debian-local/dists/xenial/main/binary-x86_64/Packages
2019-01-31T23:48:38.118Z Found remote resource with last modified time - Thu Jan 31 23:38:53 UTC 2019
2019-01-31T23:48:38.119Z Found remote resource with ETag - 371b0960dd7103e7009c744f2d3fb83264f2aa67
2019-01-31T23:48:38.119Z Found remote resource with content length - 1658
2019-01-31T23:48:38.119Z Found remote resource with checksums - [ChecksumInfo
, ChecksumInfo
{type=SHA-1, original='371b0960dd7103e7009c744f2d3fb83264f2aa67', actual='null'}, ChecksumInfo
{type=SHA-256, original='641d0a0b29630f2839a811cc4228b58ca807fd0d8dca081daf19c9e0f1836e4c', actual='null'}]
2019-01-31T23:48:38.119Z Returning found remote resource info
2019-01-31T23:48:38.119Z Requested resource is found = true
2019-01-31T23:48:38.119Z Request is HEAD = false
2019-01-31T23:48:38.119Z Request is for a checksum = false
2019-01-31T23:48:38.119Z Target repository is not remote or doesn't store locally = false
2019-01-31T23:48:38.119Z Requested resource was not modified = false
2019-01-31T23:48:38.119Z Responding with found resource
2019-01-31T23:48:38.119Z Executing any AltResponse user plugins that may exist
2019-01-31T23:48:38.119Z Alternative response status is set to -1 and message to 'null'
2019-01-31T23:48:38.119Z Found no alternative content handles
2019-01-31T23:48:38.119Z Retrieving a content handle from target repo
2019-01-31T23:48:38.119Z Resource is not yet in cache, performing download first
2019-01-31T23:48:38.119Z The requested resource isn't pre-resolved
2019-01-31T23:48:38.120Z Target repository isn't virtual - verifying that downloading is allowed
2019-01-31T23:48:38.120Z Creating a resource handle from 'debian-remote-test'
2019-01-31T23:48:38.120Z Target repository is configured to retain artifacts locally - resource will be stored and the streamed to the user
2019-01-31T23:48:38.120Z Remote repository is online
2019-01-31T23:48:38.123Z Found the resource in the cache - checking for expiry
2019-01-31T23:48:38.123Z Returning resource as expired
2019-01-31T23:48:38.124Z Force expiration on the cached resource = false
2019-01-31T23:48:38.124Z Resource isn't cached and isn't expired = false
2019-01-31T23:48:38.124Z Found expired cached resource and is newer than remote = true
2019-01-31T23:48:38.124Z Remote property synchronization is disabled - expired resource property synchronization not attempted
2019-01-31T23:48:38.124Z Un-expiring cached resource if needed
2019-01-31T23:48:38.124Z Is resource metadata = false
2019-01-31T23:48:38.124Z Un-expiring the resource
2019-01-31T23:48:38.149Z Found the resource in the cache - checking for expiry
2019-01-31T23:48:38.150Z Found request parameter {}=artifactory.forceDownloadIfNewer
2019-01-31T23:48:38.150Z Returning cached resource
2019-01-31T23:48:38.159Z Removing the resource from all failed caches
2019-01-31T23:48:38.159Z Returning the cached resource
2019-01-31T23:48:38.159Z Creating a resource handle from 'debian-remote-test-cache:dists/xenial/main/binary-x86_64/Packages'
2019-01-31T23:48:38.160Z Identified requested resource as a file
2019-01-31T23:48:38.160Z Requested resource is an ordinary artifact - using normal content handle with length '1658'
2019-01-31T23:48:38.160Z Executing any BeforeDownload user plugins that may exist
2019-01-31T23:48:38.160Z Responding with selected content handle
2019-01-31T23:48:38.161Z Request succeeded
[SHA256SUMS]
Request ID: 436cdce3
Repo Path ID: debian-remote-test:dists/xenial/main/binary-x86_64/current/images/SHA256SUMS
Method Name: GET
User: admin
Time: 2019-01-31T23:49:14.080Z
Thread: http-nio-8081-exec-10
Steps:
2019-01-31T23:49:14.081Z Received request
2019-01-31T23:49:14.081Z Request source = RESTRICTED_IP, Last modified = 31-12-69 23:59:59 +00:00, If modified since = -1, Thread name = http-nio-8081-exec-10
2019-01-31T23:49:14.081Z Executing any BeforeDownloadRequest user plugins that may exist
2019-01-31T23:49:14.081Z Retrieving info from {} repository '{}' type
2019-01-31T23:49:14.083Z Found the resource in the cache - checking for expiry
2019-01-31T23:49:14.083Z Returning cached resource
2019-01-31T23:49:14.083Z Found resource in local cache - returning cached resource
2019-01-31T23:49:14.083Z Requested resource is found = true
2019-01-31T23:49:14.083Z Request is HEAD = false
2019-01-31T23:49:14.083Z Request is for a checksum = false
2019-01-31T23:49:14.084Z Target repository is not remote or doesn't store locally = true
2019-01-31T23:49:14.084Z Requested resource was not modified = false
2019-01-31T23:49:14.084Z Responding with found resource
2019-01-31T23:49:14.084Z Executing any AltResponse user plugins that may exist
2019-01-31T23:49:14.084Z Alternative response status is set to -1 and message to 'null'
2019-01-31T23:49:14.084Z Found no alternative content handles
2019-01-31T23:49:14.084Z Retrieving a content handle from target repo
2019-01-31T23:49:14.084Z The requested resource isn't pre-resolved
2019-01-31T23:49:14.084Z Target repository isn't virtual - verifying that downloading is allowed
2019-01-31T23:49:14.084Z Creating a resource handle from 'debian-remote-test-cache:dists/xenial/main/binary-x86_64/current/images/SHA256SUMS'
2019-01-31T23:49:14.085Z Identified requested resource as a file
2019-01-31T23:49:14.085Z Requested resource is an ordinary artifact - using normal content handle with length '5'
2019-01-31T23:49:14.085Z Executing any BeforeDownload user plugins that may exist
2019-01-31T23:49:14.085Z Responding with selected content handle
2019-01-31T23:49:14.085Z Request succeeded