Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-7159

Multi-threaded Package Indexing



    • Type: Improvement
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: 3.4.0
    • Fix Version/s: 5.5.0
    • Component/s: YUM
    • Labels:
    • Environment:

      Host OS: RedHat Enterprise Linux 6.5
      HTTP Host: nginx
      JVM Host: Tomcat 7
      Application: Artifactory 3.4.0
      Number of Artifactory Repositories with yum repos: 5 (some have multiple separate yum repos)


      Our servers have multiple YUM repositories that all hold 10k+ packages. All repositories are actively being deployed to as part of a continuous delivery pipeline. We have started to notice that when we verify our package deployment via the index updating, a deployment of packages to any of our large yum repos takes sometimes 20 minutes to verify.

      After a bit of investigation of the Artifactory logs, we noticed that each repository is only taking 4-5 minutes to index, but when there's only a single thread performing the indexing for all repositories and when there are 4+ large repos, the updating of a single repo's index has to wait on the updating of all the other repositories' indexes.

      My team and I think that it would be appropriate that each index has its own thread for managing and updating its index. This same request can (and should be) applied to any of the indexing support that other repo types have, such as debian, etc.

      One option could be that the thread creation/activation could be controlled by the yum package checkbox and would be scoped to only that Artifactory repository. Once the checkbox is checked and the settings are saved, a separate thread is created for indexing that repo's yum repos. When the checkbox is unchecked, the indexing thread is stopped.

      Another option would be to have a setting that controls how many threads are created for each "indexer". That way a common yum indexer "thread pool" could be used by many Artifactory repositories for updating the yum repos they contain.


          Issue Links



              chenk Chen Keinan (Inactive)
              kleinen Kyle Leinen
              19 Vote for this issue
              12 Start watching this issue