Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-18716

Conda Virtuals never return 304 on repodata requests (Performance)

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: 4 - Normal
    • Resolution: Done
    • Affects Version/s: 6.7.0, 6.8.0, 6.5.9
    • Fix Version/s: 6.11.0
    • Component/s: Conda
    • Labels:
    • Severity:
      Medium

      Description

      SENSITIVE DATA HAS BEEN REMOVED FROM THIS TICKET ACCORDING TO JFROG'S INTERNAL POLICY.

       

      Currently, when a user makes a "conda search" or other Conda client action against Artifactory, the system sends back the full repodata.json file even if no changes had been made.

       

      This reduces performance, as the conda client has to re-process the JSON every time a request is made.

       

      Steps to reproduce:

      1. Point Conda at a virtual Artifactory Conda repository
      2. Run a "conda search" or "conda install" twice, note the time the request takes:6

      time conda search 'scipy'

      Loading channels: done

      1. Name Version Build Channel
        scipy 0.11.0 np15py26_2 artifactory/api/conda/conda-virt

      [...]
      real 0m26.725s
      user 0m22.615s
      sys 0m0.990s

      #second run
      real 0m17.606s
      user 0m15.168s
      sys 0m0.813s

      3. Remove "/api/conda" from the ~/.condarc file, bypassing the Artifactory Conda API implementation

      4. Re-run the search, note the great performance improvements:

      time conda search 'scipy

       # Name Version Build Channel
      scipy 0.11.0 np15py26_2 artifactory/conda-virtual
      [...]
      real 0m7.162s
      user 0m6.836s
      sys 0m0.263s

      #Second run
      real 0m1.281s
      user 0m1.041s
      sys 0m0.220s

      From the Artifactory request logs, it looks like when "api/conda" is bypassed, a 304 Not Modified is sent if the files haven't changed:

      [1st search - No /api/conda]
      20190308191513|126|REQUEST|RESTRICTED_IP|admin|GET|/conda-generic-test/pkgs/pro/noarch/repodata.json|HTTP/1.1|200|101
      [2nd search - No /api/conda]
      20190308191608|10|REQUEST|RESTRICTED_IP|admin|GET|/conda-generic-test/pkgs/pro/noarch/repodata.json|HTTP/1.1|304|0

       

      [1st search - /api/conda]
      20190308193146|2507|REQUEST|RESTRICTED_IP|admin|GET|/api/conda/conda-virt/linux-64/repodata.json|HTTP/1.1|200|11666217
      [2nd search - /api/conda]
      20190308193714|1606|REQUEST|RESTRICTED_IP|admin|GET|/api/conda/conda-virt/linux-64/repodata.json|HTTP/1.1|200|11666217

       

      Conda works fine without /api/conda, and has bad performance while it is used because of these repeated requests for the same information.

       

      Artifactory should send back a "304" response if the client already has the repodata file, similar to how it handles generic requests.

       

        Attachments

          Activity

            People

            Assignee:
            Dudim Dudi Morad (Inactive)
            Reporter:
            patrickr Patrick Russell
            Votes:
            1 Vote for this issue
            Watchers:
            7 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Sync Status

                Connection: RTFACT Sync
                RTMID-18716 -
                SYNCHRONIZED
                • Last Sync Date: