Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-13856

Pip doesn't always download from local cache with Artifactory as package index



    • Severity:


      It seems like pip doesn't download from its local cache for some packages when using an Artifactory pypi repository as its custom package index.

      To reproduce:
      -install a package (The package i used was pyOpenSSL) from the python library (https://pypi.python.org) with pip install <package>. Take note that the wheels are being cached and of the location under ~/.cache/pip/.

      -uninstall the package with pip uninstall <package>. Then install it again. It will say:
      Collecting <package>
      Using cached <package>

      -Create a virtual pypi repository using https://pypi.python.org as the url. Add the repository url as the index-url in ~/.pip/pip.conf:
      index-url = http://admin:password@

      -uninstall and re-install the package again using pip install <package> --trusted-host <repository IP>. Notice how it will download the package even though it was cached earlier when installing from https://pypi.python.org.

      -Uninstall the package and reinstall it again. The package will never install from the local cache located at ~/.cache/pip when resolving from Artifactory.

      The issue may have to do with the Artifactory headers:
      Artifactory Headers:

      HTTP/1.1 200 OK Date: Fri, 03 Mar 2017 22:17:23 GMT Content-Type: application/octet-stream Content-Length: 43918 Connection: keep-alive Server: Artifactory/5.1.0 X-Artifactory-Id: aolshared1f-leadpages Last-Modified: Sat, 15 Oct 2016 13:16:24 GMT ETag: c0db373477cfe46ddad0e9af45a5f5d3ffa76a4e X-Checksum-Sha1: c0db373477cfe46ddad0e9af45a5f5d3ffa76a4e Accept-Ranges: bytes X-Checksum-Md5: 7c87cf718171f736f29d1becb4c7b7a5 X-Artifactory-Filename: pyOpenSSL-16.2.0-py2.py3-none-any.whl Content-Disposition: attachment; filename="pyOpenSSL-16.2.0-py2.py3-none-any.whl"; filename*=UTF-8''pyOpenSSL-16.2.0-py2.py3-none-any.whl X-Node: nginx-shared1f

      pypi.python.org headers:

      HTTP/1.1 200 OK x-amz-replication-status: COMPLETED Last-Modified: Sat, 15 Oct 2016 13:16:24 GMT ETag: "7c87cf718171f736f29d1becb4c7b7a5" x-amz-meta-version: 16.2.0 x-amz-meta-project: pyopenssl x-amz-meta-python-version: py2.py3 x-amz-meta-package-type: bdist_wheel Content-Type: binary/octet-stream Server: AmazonS3 Via: 1.1 varnish Fastly-Debug-Digest: 47d3aea12e9c9cafc81afbdd9196cc9ce4dce1929b553605b49f12a3b0feee69 Cache-Control: max-age=31557600, public Content-Length: 43918 Accept-Ranges: bytes Date: Fri, 03 Mar 2017 22:19:15 GMT Via: 1.1 varnish Age: 2552281 Connection: keep-alive X-Served-By: cache-sea1026-SEA, cache-msp9222-MSP X-Cache: HIT, HIT X-Cache-Hits: 7443, 1 X-Timer: S1488579555.375859,VS0,VE2 X-Frame-Options: deny X-XSS-Protection: 1; mode=block X-Content-Type-Options: nosniff X-Permitted-Cross-Domain-Policies: none

      With the important header being `Cache-Control: max-age=31557600`. Since the max-age isn't being set pip is assuming that the package isn't fresh so it invalidates the cache.

      Here's the code in the library that pip is using which is causing local caches to be ignored



          Issue Links



                liorg Lior Gur (Inactive)
                mattheww Matthew Wang
                8 Vote for this issue
                13 Start watching this issue



                    Sync Status

                    Connection: RTFACT Sync
                    RTMID-13856 -
                    • Last Sync Date: