Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-22966

RPM virtual repository is listing packages only from the first repository from the added list

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: 3 - High
    • Resolution: Done
    • Affects Version/s: 6.12.0, 6.11.6, 6.18.0, 6.18.1, 7.6.3
    • Fix Version/s: 7.12.3
    • Component/s: RPM
    • Labels:
      None
    • Severity:
      High
    • Regression:
      Yes

      Description

      Issue description: RPM virtual repository is listing packages only from the first repository in the list added to virtual repository. 

      Expected behavior: It should add both(or all) the counts from each repository and display in yum list. 

      Replicatoon steps:
      STEP1:

      Create an rpm remote repository rpm-remote1 and point to https://rpm.nodesource.com/pub_6.x/el/7/x86_64/
      Create another rpm remote repository rpm-remote2 and point to http://openresty.org/package/centos/7/x86_64/

      STEP2

      Create a virtual repository rpm-virtual and configure in rpm client to point to virtual as below. 

      $ sudo vi /etc/yum.repos.d/artifactory.repo

      [Artifactory]
      name=Artifactory
      baseurl=http://user:password@localhost:8081/artifactory/rpm-virtual
      enabled=1
      gpgcheck=0

      STEP3:

      Now add only one rpm-remote1 to virtual. 

      From rpm client machine, run the below set of commands. 

      $ sudo yum clean all
      $ yum --disablerepo="*" --enablerepo="Artifactory" list available | wc -l
      7  (The count is 7)

      STEP4:

      Remove the earlier added rpm-remote1 and add only rpm-rermote2 to the virtual. Zap cache virtual. 

      From rpm client machine, run the below set of commands. 

      $ sudo yum clean all
      $ yum --disablerepo="*" --enablerepo="Artifactory" list available | wc -l
      48 (The count is 48 and it is different from the value in Step3)

      STEP5: 

      Now add both remote repositories rpm-remote2 as first repository and rpm-remote1 as second repository in the list, to Virtual repository and run the same command. 

      $ sudo yum clean all
      $ }}{{yum --disablerepo="*" --enablerepo="Artifactory" list available | wc -l{{}}
      48 (This count is not an aggregation of above two remotes, rather it is showing the count only from the first repository in the list)

      Log Analysis:  (Below is the log from Artifactory 7.6.3)

      When there is only one remote repository in the virtual list, below is the DEBUG log. 

      2020-08-12 13:42:35,693 [http-nio-8081-exec-2] [TRACE] (o.a.a.y.i.YumVirtualInterceptor:47) - Handling cached resource interception for repo rpm-virtual and path repodata/repomd.xml
      2020-08-12 13:42:35,694 [http-nio-8081-exec-2] [DEBUG] (o.a.a.y.i.YumVirtualInterceptor:54) - yum virtual repo rpm-virtual cache for path repodata/repomd.xml stale or does not exist.
      2020-08-12 13:42:35,694 [http-nio-8081-exec-2] [TRACE] (o.a.a.y.i.YumVirtualInterceptor:66) - Handling getInfo interception for repo rpm-virtual and path repodata/repomd.xml
      2020-08-12 13:42:35,695 [http-nio-8081-exec-2] [DEBUG] (o.a.a.y.i.YumVirtualInterceptor:68) - Repo rpm-virtual aggregates one or less repositories, yum virtual metadata calculation will not trigger.

      Now after adding one more remote repository to virtual, below is the DEBUG log. 

      2020-08-12 13:45:52,708 [http-nio-8081-exec-6] [TRACE] (o.a.a.y.i.YumVirtualInterceptor:47) - Handling cached resource interception for repo rpm-virtual and path repodata/repomd.xml2020-08-12 13:45:52,708 [http-nio-8081-exec-6] [TRACE] (o.a.a.y.i.YumVirtualInterceptor:47) - Handling cached resource interception for repo rpm-virtual and path repodata/repomd.xml2020-08-12 13:45:52,708 [http-nio-8081-exec-6] [DEBUG] (o.a.a.y.i.YumVirtualInterceptor:54) - yum virtual repo rpm-virtual cache for path repodata/repomd.xml stale or does not exist.2020-08-12 13:45:52,709 [http-nio-8081-exec-6] [TRACE] (o.a.a.y.i.YumVirtualInterceptor:66) - Handling getInfo interception for repo rpm-virtual and path repodata/repomd.xml2020-08-12 13:45:52,710 [http-nio-8081-exec-6] [INFO ] (o.a.a.y.i.YumVirtualInterceptor:80) - Triggering synchronous virtual yum index calculation for path repodata/repomd.xml in repo rpm-virtual2020-08-12 13:45:52,710 [art-exec-8] [DEBUG] (o.a.a.y.v.i.YumVirtualCalculatorImpl:52) - Starting virtual yum metadata calculation for path rpm-virtual/repodata/2020-08-12 13:45:52,711 [art-exec-8] [DEBUG] (o.a.a.y.v.i.YumVirtualIndexesCollector:106) - Testing for changes in repomd.xml in all of rpm-virtual's aggregated repos: [rpm-remote1, rpm-remote2]

      2020-08-12 13:45:52,713 [art-exec-8] [TRACE] (o.a.a.y.v.i.YumVirtualIndexesCollector:187) - Found new index at rpm-remote1:repodata/repomd.xml

      2020-08-12 13:45:52,714 [art-exec-8] [TRACE] (o.a.a.y.v.i.YumVirtualIndexesCollector:187) - Found new index at rpm-remote2:repodata/repomd.xml

      2020-08-12 13:45:52,714 [art-exec-8] [DEBUG] (o.a.a.y.v.i.YumVirtualIndexesCollector:157) - No properties found on path rpm-virtual/repodata/repomd.xml to compare with aggregated resources - forcing calculation on path rpm-virtual/repodata/2020-08-12 13:45:52,714 [art-exec-8] [INFO ] (o.a.a.y.v.i.YumVirtualIndexesCollector:74) - Calculation of virtual yum metadata on path repodata required, collecting all indexes under path of repos aggregated by rpm-virtual2020-08-12 13:45:52,715 [art-exec-8] [DEBUG] (o.a.a.y.v.i.YumVirtualIndexesCollector:302) - Reading content of repomd.xml at rpm-remote1/repodata/repomd.xml2020-08-12 13:45:52,717 [art-exec-8] [DEBUG] (o.a.a.y.v.r.YumIndexPathsResolver:58) - Resolving index paths from repomd.xml at rpm-remote1/repodata/2020-08-12 13:45:53,345 [art-exec-8] [DEBUG] (o.a.a.y.v.i.YumVirtualIndexesCollector:227) -org.artifactory.addon.yum.virtual.index.exception.YumVirtualIndexSkipRepo: Path repodata/8182c4ad4f0a0564900736a2d06a24b2700e2f34-primary.xml.gz does not exists in repo rpm-remote1, skipping entire index under path. repodata for this repo at org.artifactory.addon.yum.virtual.index.YumVirtualIndexesCollector.getIndex(YumVirtualIndexesCollector.java:265) at org.artifactory.addon.yum.virtual.index.YumVirtualIndexesCollector.collectIndexesFromAggregatedRepos(YumVirtualIndexesCollector.java:221) at org.artifactory.addon.yum.virtual.index.YumVirtualIndexesCollector.collect(YumVirtualIndexesCollector.java:76) at org.artifactory.addon.yum.virtual.index.YumVirtualCalculatorImpl.calculateYumVirtualMetadata(YumVirtualCalculatorImpl.java:68) at org.artifactory.addon.yum.virtual.index.YumVirtualCalculatorImpl.calculateYumVirtualMetadata(YumVirtualCalculatorImpl.java:47) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.artifactory.work.queue.WorkQueueImpl.invoke(WorkQueueImpl.java:123) at org.artifactory.work.queue.WorkQueueImpl.doJobs(WorkQueueImpl.java:96) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at org.artifactory.opentracing.TraceableRunnableDecorator.run(TraceableRunnableDecorator.java:30) at org.artifactory.concurrent.ArtifactoryRunnable.run(ArtifactoryRunnable.java:53) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834)2020-08-12 13:45:53,347 [art-exec-8] [DEBUG] (o.a.a.y.v.i.YumVirtualIndexesCollector:302) - Reading content of repomd.xml at rpm-remote2/repodata/repomd.xml2020-08-12 13:45:53,349 [art-exec-8] [DEBUG] (o.a.a.y.v.r.YumIndexPathsResolver:58) - Resolving index paths from repomd.xml at rpm-remote2/repodata/2020-08-12 13:45:53,459 [art-exec-8] [TRACE] (o.a.a.y.v.i.YumVirtualIndexesCollector:281) - Adding path rpm-remote2/repodata/4d2ac0c47bab4503843c8a71eea6bb90adee138a83dbd26def9f58da18f005c9-primary.xml.gz to primary.xml merge list2020-08-12 13:45:53,460 [art-exec-8] [TRACE] (o.a.a.y.v.i.YumVirtualIndexesCollector:281) - Adding path rpm-remote2/repodata/35286f051e365c93dafbf97fb35c03f0962f061ee9b81f87b275bba862efd0ea-filelists.xml.gz to filelists.xml merge list2020-08-12 13:45:53,460 [art-exec-8] [TRACE] (o.a.a.y.v.i.YumVirtualIndexesCollector:281) - Adding path rpm-remote2/repodata/d10fe9b43d138566b2a5f5148cb435af0cafac97bb8f9af51b881644dd99c80b-other.xml.gz to other.xml merge list2020-08-12 13:45:53,460 [art-exec-8] [TRACE] (o.a.a.y.v.i.YumVirtualIndexesCollector:336) - Found checksum 4791f2e7f84f2087b150a9cf9e6b0906152d7e77 for repomd at path rpm-remote2/repodata/repomd.xml2020-08-12 13:45:53,460 [art-exec-8] [DEBUG] (o.a.a.y.v.i.YumVirtualIndexesCollector:89) - Virtual repo rpm-virtual found 1 index locations under path repodata in all aggregated repos, no need to calculate metadata.2020-08-12 13:45:53,460 [art-exec-8] [DEBUG] (o.a.a.y.v.i.YumVirtualCalculatorImpl:93) - Unexpiring all indexes under path rpm-virtual/repodata/

      The aggregation is failing when there are more than one repository under a virtual. 

        Attachments

          Activity

            People

            Assignee:
            igoru Igor Usenko [EXT]
            Reporter:
            gajapathik Gajapathi Kimidi
            Votes:
            3 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Sync Status

                Connection: RTFACT Sync
                RTMID-22966 -
                SYNCHRONIZED
                • Last Sync Date: