Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-21666

NPM search HTTP connection leak when failing to parse the response

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Done
    • Priority: 2 - Critical
    • Resolution: Done
    • Affects Version/s: 6.8.7, 6.18.0, 7.3.0, 6.18.1
    • Fix Version/s: 7.5.0, 6.20.0
    • Component/s: None
    • Labels:
      None
    • Severity:
      Critical

      Description

      There is a connection leak in case NPM search is performed, and the response from the remote repo wasn't a JSON.

      Reproduction Steps
      1. Configure Artifactory with a proxy. I used "Charles" proxy so I can configure breakpoints.
      2. Configure the proxy to be used on npm-remote repository.
      3. Configure a breakpoint in the proxy to alter responses from "registry.npmjs.org".
      4. Perform an "npm search foo". Catch the response in the proxy and change the returned data from JSON to HTML.
      5. Connect to the JMX MBean of "org.jfrog.artifactory;Artifactory;HTTPConnectionPool;<REPO-NAME>;Attributes;Leased" and see it is increased.
      6. Every such action will increase the Leased connections by one.

      Once the limit of 50 connections is reached, Artifactory won't go out for any NPM request and the threads will hang.
      Once we have passed 200 requests the entire Artifactory service will hang as it ran out of HTTP connections.

      In a thread dumps all threads will be "stuck" like this:

      "http-nio-8081-exec-8" Id=42 in WAITING on lock=java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject@185568d7
          at sun.misc.Unsafe.park(Native Method)
          at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
          at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
          at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:138)
          at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:306)
          at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64)
          at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192)
          at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185)
          at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107)
          at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276)
          at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263)
          at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190)
          at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
          at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
          at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
          at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
          at org.jfrog.client.http.CloseableHttpClientDecorator.doExecute(CloseableHttpClientDecorator.java:107)
          at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
          at org.artifactory.repo.HttpRepo.doExecuteMethod(HttpRepo.java:489)
          at org.artifactory.repo.HttpRepo.doExecuteMethod(HttpRepo.java:483)
          at org.artifactory.repo.HttpRepo.executeMethod(HttpRepo.java:505)
          at org.artifactory.repo.HttpRepo.retrieveInfo(HttpRepo.java:537)
          at org.artifactory.repo.RemoteRepoBase.getRemoteResource(RemoteRepoBase.java:422)
          at org.artifactory.repo.RemoteRepoBase.internalGetInfo(RemoteRepoBase.java:340)
          at org.artifactory.repo.RemoteRepoBase.getInfo(RemoteRepoBase.java:290)
          at org.artifactory.addon.npm.repo.NpmRemoteRepoHandler.getPackageMetadata(NpmRemoteRepoHandler.java:121)
          at org.artifactory.addon.npm.repo.merge.NpmPackageMetadataMerger.merge(NpmPackageMetadataMerger.java:92)
          at org.artifactory.addon.npm.repo.merge.NpmMetadataMerger.getMergedResult(NpmMetadataMerger.java:67)
          at org.artifactory.addon.npm.repo.NpmVirtualRepoHandler.getPackageMetadata(NpmVirtualRepoHandler.java:63)
          at org.jfrog.repomd.npm.rest.NpmSubResource.packageInfo(NpmSubResource.java:62)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:76)
          at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$$Lambda$834/2092005749.invoke(Unknown Source)
          at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:148)
          at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:191)
          at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ResponseOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:200)
          at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:103)
          at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:493)
          at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:415)
          at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:104)
          at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:277)
          at org.glassfish.jersey.internal.Errors$1.call(Errors.java:272)
          at org.glassfish.jersey.internal.Errors$1.call(Errors.java:268)
          at org.glassfish.jersey.internal.Errors.process(Errors.java:316)
          at org.glassfish.jersey.internal.Errors.process(Errors.java:298)
          at org.glassfish.jersey.internal.Errors.process(Errors.java:268)
          at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:289)
          at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:256)
          at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:703)
          at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:416)
          at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:370)
          at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:389)
          at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:342)
          at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:229)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
          at org.artifactory.webapp.servlet.RepoFilter.execute(RepoFilter.java:186)
          at org.artifactory.webapp.servlet.RepoFilter.doFilter(RepoFilter.java:96)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
          at org.artifactory.webapp.servlet.AccessFilter.useAuthentication(AccessFilter.java:426)
          at org.artifactory.webapp.servlet.AccessFilter.authenticateAndExecute(AccessFilter.java:305)
          at org.artifactory.webapp.servlet.AccessFilter.doFilterInternal(AccessFilter.java:208)
          at org.artifactory.webapp.servlet.AccessFilter.doFilter(AccessFilter.java:167)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
          at org.artifactory.webapp.servlet.RequestFilter.doFilter(RequestFilter.java:77)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
          at org.artifactory.webapp.servlet.ArtifactoryCsrfFilter.doFilter(ArtifactoryCsrfFilter.java:74)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
          at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:164)
          at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:80)
          at org.artifactory.webapp.servlet.SessionFilter.doFilter(SessionFilter.java:62)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
          at org.artifactory.webapp.servlet.ArtifactoryFilter.doFilter(ArtifactoryFilter.java:124)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
          at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
          at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:279)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
          at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)
          at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
          at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:800)
          at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1471)
          at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
            - locked org.apache.tomcat.util.net.NioEndpoint$NioSocketWrapper@2726bcb4
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
          at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            arielk Ariel Kabov
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Sync Status

                Connection: RTFACT Sync
                RTMID-21666 -
                SYNCHRONIZED
                • Last Sync Date: