Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-20590

compress debian metadata types in parallel rather than serially

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Currently, each archive is done serially, rather than in parallel, which can lead to longer indexing times for larger environments - see bz2 taking 1 minute and 40 seconds. This can propagate to many minutes the more nodes there are:

      2019-10-28 23:39:51,867 [art-exec-427924] [DEBUG] (o.j.r.d.i.a.DebianAutomaticRepoMetadataIndexer:117) - Writing Packages to path dists/xenial/temp-1572305948526/xenial/main/binary-amd64/Packages for coordinates xenial/main/amd64
      2019-10-28 23:40:03,455 [art-exec-427924] [DEBUG] (o.a.a.d.i.DebianLocalInterceptor:94) - Adding sha256 to Packages file at debian-local/dists/xenial/temp-1572305948526/xenial/main/binary-amd64/Packages
      2019-10-28 23:40:03,470 [art-exec-427924] [DEBUG] (o.j.r.d.i.a.DebianAutomaticRepoMetadataIndexer:117) - Writing Packages.gz to path dists/xenial/temp-1572305948526/xenial/main/binary-amd64/Packages.gz for coordinates xenial/main/amd64
      2019-10-28 23:40:13,292 [art-exec-427924] [DEBUG] (o.a.a.d.i.DebianLocalInterceptor:94) - Adding sha256 to Packages file at debian-local/dists/xenial/temp-1572305948526/xenial/main/binary-amd64/Packages.gz
      2019-10-28 23:40:13,302 [art-exec-427924] [DEBUG] (o.j.r.d.i.a.DebianAutomaticRepoMetadataIndexer:117) - Writing Packages.bz2 to path dists/xenial/temp-1572305948526/xenial/main/binary-amd64/Packages.bz2 for coordinates xenial/main/amd64
      2019-10-28 23:41:53,148 [art-exec-427924] [DEBUG] (o.a.a.d.i.DebianLocalInterceptor:94) - Adding sha256 to Packages file at debian-local/dists/xenial/temp-1572305948526/xenial/main/binary-amd64/Packages.bz2
      

      We should attempt to do these in parallel to reduce indexing times.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              loreny Loren Yeung
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: