Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-17688

Debian Incremental Indexer performance during filteration of removed entries can be improved

    Details

    • Type: Performance
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Debian
    • Labels:
      None

      Description

      Our code scans and filters the current stored version of the Packages file by version-blocks (multiline metadata blocks of individual versions) block by block, filtering files that have been deleted since the last calculation. 

      There are two key factors in this process where it can be improved:

      1. It is enough to delete/override a single file to have our code scan the entire Packages file. Suggested: find a way to make this an o(1) operation. Either the deleted file has a block in the current Packages file or it doesn't. The problem is that we build the new Packages file as we iterate throughout all the blocks in the Packages file, and modelling the Packages file is not a trivial task if other solutions are to be considered.
      2. If fixing #1 is not feasible, consider passing just the filename of to regex matcher rather than the entire metadata block, making the regex computation easier. 

      My local test shows that on a 3K packages Packages file, this process alone, when overriding a single deb file, takes 7 seconds.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              uriahl Uriah Levy
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: