Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-19421

Nuget remote can incorrectly cache V2 XML metadata as feed.json resulting in packages' resolution failures

    Details

    • Type: Bug
    • Status: Open
    • Priority: High
    • Resolution: Unresolved
    • Affects Version/s: 6.11.0
    • Fix Version/s: None
    • Component/s: NuGet
    • Labels:
      None
    • Regression:
      Yes

      Description

      Summary:

      Trying to define remote repo with Nuget V3 API against another Artifactory (documentation here), while using non-V3 Feed URL endpoint, will incorrectly cache a <repoKey>/.nuGetV3/feed.json file with Nuget V2 Xml content (screenshots attached). This will result in nuget packages' resolution failures with the following below.

      Expected V3 Feed URL:
      <artifactory_url>/artifactory/api/nuget/v3/nugetRepoKey

      Culprit V3 Feed URL:
      <artifactory_url>/artifactory/api/nuget/nugetRepoKey

      • It is possible to run into this scenario when trying to configure V3 for the first time.
      • Changing the Nuget Feed V3 to be correct - will not be sufficient if you try to workaround this. You will need to delete the malformed feed.json (workaround detailed below)
      nuget install Microsoft.AspNetCore.Mvc.Formatters.Json -Verbosity detailed
      

      Client side:

      NuGet Version: 4.7.0.5148
      Feeds used:
        http://localhost:8080/artifactory/api/nuget/v3/nuget-cache-v2
      
      Installing package 'Microsoft.AspNetCore.Mvc.Formatters.Json' to '/Users/andreik/JFrog/Artifactory/SourceCode/forks'.
      Unable to load the service index for source http://localhost:8080/artifactory/api/nuget/v3/nuget-cache-v2.
        404 ()
      NuGet.Protocol.Core.Types.FatalProtocolException: Unable to load the service index for source http://localhost:8080/artifactory/api/nuget/v3/nuget-cache-v2. ---> System.Net.Http.HttpRequestException: 404 ()
        at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode () [0x0002a] in <935151bbf04949bba6171dd2141bd9d1>:0
        at NuGet.Protocol.HttpSource+<>c__DisplayClass14_0`1[T].<GetAsync>b__0 (System.Threading.CancellationToken lockedToken) [0x00400] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.Common.ConcurrencyUtilities.ExecuteWithFileLockedAsync[T] (System.String filePath, System.Func`2[T,TResult] action, System.Threading.CancellationToken token) [0x00223] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.Protocol.HttpSource.GetAsync[T] (NuGet.Protocol.HttpSourceCachedRequest request, System.Func`2[T,TResult] processAsync, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x000ea] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.Protocol.ServiceIndexResourceV3Provider.GetServiceIndexResourceV3 (NuGet.Protocol.Core.Types.SourceRepository source, System.DateTime utcNow, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x001ff] in <55431ab4846b49babe0aafcf24cd67ed>:0
         --- End of inner exception stack trace ---
        at NuGet.Protocol.ServiceIndexResourceV3Provider.GetServiceIndexResourceV3 (NuGet.Protocol.Core.Types.SourceRepository source, System.DateTime utcNow, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x002b3] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.Protocol.ServiceIndexResourceV3Provider.TryCreate (NuGet.Protocol.Core.Types.SourceRepository source, System.Threading.CancellationToken token) [0x00213] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync[T] (System.Threading.CancellationToken token) [0x000b8] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.Protocol.DependencyInfoResourceV3Provider.TryCreate (NuGet.Protocol.Core.Types.SourceRepository source, System.Threading.CancellationToken token) [0x00082] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync[T] (System.Threading.CancellationToken token) [0x000b8] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.Protocol.Core.Types.SourceRepository.GetResourceAsync[T] () [0x00070] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.PackageManagement.NuGetPackageManager.GetLatestVersionCoreAsync (System.String packageId, NuGet.Frameworks.NuGet.Frameworks1094018.NuGetFramework framework, NuGet.PackageManagement.ResolutionContext resolutionContext, NuGet.Protocol.Core.Types.SourceRepository source, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x0008c] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.PackageManagement.NuGetPackageManager+<>c__DisplayClass92_1.<GetLatestVersionAsync>b__3 () [0x000a7] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.PackageManagement.NuGetPackageManager.GetLatestVersionAsync (System.String packageId, NuGet.Frameworks.NuGet.Frameworks1094018.NuGetFramework framework, NuGet.PackageManagement.ResolutionContext resolutionContext, System.Collections.Generic.IEnumerable`1[T] sources, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x00115] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.PackageManagement.NuGetPackageManager.GetLatestVersionAsync (System.String packageId, NuGet.ProjectManagement.NuGetProject project, NuGet.PackageManagement.ResolutionContext resolutionContext, System.Collections.Generic.IEnumerable`1[T] sources, NuGet.Common.ILogger log, System.Threading.CancellationToken token) [0x000a5] in <55431ab4846b49babe0aafcf24cd67ed>:0
        at NuGet.CommandLine.InstallCommand.InstallPackageAsync (System.String packageId, NuGet.Versioning.NuGetVersion version, System.String installPath) [0x00229] in <55431ab4846b49babe0aafcf24cd67ed>:0
      

      Artifactory (remote repo) side:

      2019-06-17 19:30:07,836 [http-nio-8080-exec-10] INFO  o.a.r.HttpRepo:432 - nuget-cache-v2 downloading http://localhost:8081/artifactory/api/nuget/nuget-test Unknown content length 
      2019-06-17 19:30:07,842 [http-nio-8080-exec-10] INFO  o.a.r.HttpRepo:445 - nuget-cache-v2 downloaded  http://localhost:8081/artifactory/api/nuget/nuget-test 555 bytes at 86.66 KB/sec
      2019-06-17 19:30:07,844 [http-nio-8080-exec-10] ERROR o.a.a.n.r.NuGetV3VirtualAndRemoteCommon:92 - Failed to parse feed for repo: nuget-cache-v2, at url: http://localhost:8081/artifactory/api/nuget/nuget-test
      org.jfrog.common.JsonParsingException: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
       at [Source: (String)"<?xml version="1.0" encoding="utf-8"?>
      <!--
        ~
        ~ Copyright 2016 JFrog Ltd. All rights reserved.
        ~ JFROG PROPRIETARY/CONFIDENTIAL. Use is subject to license terms.
        -->
      
      <service xmlns="http://www.w3.org/2007/app" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:app="http://www.w3.org/2007/app" xml:base="http://localhost:8081/artifactory/api/nuget/nuget-test">
          <workspace>
              <atom:title>Default</atom:title>
              <collection href="Packages">
                  <atom:title>Packages</atom:t"[truncated 55 chars]; line: 1, column: 2]
      	at org.jfrog.common.MapperUtilsBase.unchecked(MapperUtilsBase.java:238)
      	at org.jfrog.common.MapperUtilsBase.readValue(MapperUtilsBase.java:120)
      	at org.jfrog.common.JsonUtils.readValue(JsonUtils.java:27)
      	at org.artifactory.addon.nugetv3.repo.NuGetV3VirtualAndRemoteCommon.downloadRemoteFeed(NuGetV3VirtualAndRemoteCommon.java:87)
      	at org.artifactory.addon.nugetv3.repo.NuGetV3RemoteRepoHandler.feed(NuGetV3RemoteRepoHandler.java:48)
      	at org.jfrog.repomd.nugetv3.rest.NuGetV3SubResource.getRoot(NuGetV3SubResource.java:40)
      	at sun.reflect.GeneratedMethodAccessor412.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
      	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
      	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
      	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200)
      	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
      	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
      	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
      	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
      	at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
      	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
      	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
      	at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
      	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
      	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
      	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
      	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
      	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
      	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
      	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
      	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.artifactory.webapp.servlet.RepoFilter.execute(RepoFilter.java:191)
      	at org.artifactory.webapp.servlet.RepoFilter.doFilter(RepoFilter.java:97)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.artifactory.webapp.servlet.AccessFilter.useAuthentication(AccessFilter.java:427)
      	at org.artifactory.webapp.servlet.AccessFilter.useAnonymousIfPossible(AccessFilter.java:392)
      	at org.artifactory.webapp.servlet.AccessFilter.doFilterInternal(AccessFilter.java:210)
      	at org.artifactory.webapp.servlet.AccessFilter.doFilter(AccessFilter.java:167)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.artifactory.webapp.servlet.RequestFilter.doFilter(RequestFilter.java:77)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.artifactory.webapp.servlet.ArtifactoryCsrfFilter.doFilter(ArtifactoryCsrfFilter.java:74)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:164)
      	at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:80)
      	at org.artifactory.webapp.servlet.SessionFilter.doFilter(SessionFilter.java:62)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.artifactory.webapp.servlet.ArtifactoryFilter.doFilter(ArtifactoryFilter.java:124)
      	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
      	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
      	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
      	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
      	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
      	at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:279)
      	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
      	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
      	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
      	at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:542)
      	at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:685)
      	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
      	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)
      	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
      	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:800)
      	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1471)
      	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: com.fasterxml.jackson.core.JsonParseException: Unexpected character ('<' (code 60)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
       at [Source: (String)"<?xml version="1.0" encoding="utf-8"?>
      <!--
        ~
        ~ Copyright 2016 JFrog Ltd. All rights reserved.
        ~ JFROG PROPRIETARY/CONFIDENTIAL. Use is subject to license terms.
        -->
      
      <service xmlns="http://www.w3.org/2007/app" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:app="http://www.w3.org/2007/app" xml:base="http://localhost:8081/artifactory/api/nuget/nuget-test">
          <workspace>
              <atom:title>Default</atom:title>
              <collection href="Packages">
                  <atom:title>Packages</atom:t"[truncated 55 chars]; line: 1, column: 2]
      	at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
      	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:693)
      	at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:591)
      	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleOddValue(ReaderBasedJsonParser.java:1902)
      	at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:757)
      	at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4141)
      	at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4000)
      	at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3004)
      	at org.jfrog.common.MapperUtilsBase.lambda$readValue$7(MapperUtilsBase.java:120)
      	at org.jfrog.common.MapperUtilsBase.unchecked(MapperUtilsBase.java:236)
      	... 75 common frames omitted
      2019-06-17 19:30:07,845 [http-nio-8080-exec-10] ERROR o.a.a.n.r.NuGetV3VirtualAndRemoteCommon:458 - Parsed feed is null, in repo: nuget-cache-v2
      

      Steps to reproduce:

      1. Setup two Artifactory instances, have Artifactory (1) created with the default Nuget remote settings pointing to Nuget Gallery (e.g. nuget-test)

      2. Have Artifactory (2)'s remote e.g. nuget-cache-v2 pointing to Aritfactory (1)'s nuget-test remote with the Artifactory remote URL set to <artifactory(1)url>/api/nuget/nuget-test with the NuGet Download Context Path set (correctly) to Download and the NuGet v3 Feed URL pointing to <artifactory(1)url>/api/nuget/nuget-test (same as the repo's URL)

      3. Setup your NuGet.Config pointing to Artifactory (2)'s nuget-cache-v2_ with V3 config:

      <add key="Artifactory1" value="http://localhost:8081/artifactory/api/nuget/v3/nuget-cache-v2" protocolVersion="3" />
      
      • The Nuget client will try to do V3 requests to the defined remote.

      4. Issue the below resolution command:

      nuget install Microsoft.AspNetCore.Mvc.Formatters.Json -Verbosity detailed

      And watch the failure as per the examples above.

      Workaround:

      1. Remove the nuget-cache-v2/.nuGetV3/feed.json file
      2. Adjust the Remote repository's V3 Feed URL to point to <artifactory_url>/artifactory/api/nuget/v3/nuget-test (replace nuget-test with your remote's name)

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              andreik Andrei Komarov
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: