docker push to Artifactory fail with errors like "Executor [org.artifactory.schedule.ArtifactoryConcurrentExecutor@3ad754b4] did not accept task"

Artifactory behind nginx.

When pushing an image to Artifactory, the process bar show progress, but at the end it fail and retry after 15 seconds.

On similar test server (But not as busy) everything works fine.

After restarting Artifactory service, pushes worked fine for a while, but the it stopped.

Error on docker workstation show HTTP Error 500.

Log from Artifactory:

2019-11-25 13:38:37,494 [http-nio-8081-exec-16] [INFO ] (o.a.e.UploadServiceImpl:376) - Deploy to 'docker-voice-ai:hello-world/_uploads/1bcd1055-39c5-4e64-89fd-03afe4f3d132' Content-Length: unspecified
2019-11-25 13:38:37,544 [http-nio-8081-exec-16] [WARN ] (o.a.s.ArtifactoryConcurrentExecutor:71) - Task org.artifactory.concurrent.ArtifactoryRunnable@5d379529 was rejected by scheduler: Task org.artifactory.concurrent.ArtifactoryRunnable@5d379529 rejected from java.util.concurrent.ThreadPoolExecutor@75223fbf[Running, pool size = 16, active threads = 16, queued tasks = 10000, completed tasks = 183940]
2019-11-25 13:38:37,545 [http-nio-8081-exec-16] [ERROR] (o.a.s.a.AsyncAdvice :299) - Task ReflectiveMethodInvocation: public abstract void org.artifactory.addon.xray.indexer.XrayEventHandler.addArtifactEvent(org.artifactory.fs.FileInfo,java.lang.String,boolean) throws java.io.IOException; target is of class [org.artifactory.addon.xray.indexer.XrayEventHandlerImpl] rejected by scheduler: Executor [org.artifactory.schedule.ArtifactoryConcurrentExecutor@3ad754b4] did not accept task: org.artifactory.schedule.aop.AsyncAdvice$$Lambda$647/904282284@25a5125e
2019-11-25 13:38:37,827 [http-nio-8081-exec-11] [WARN ] (o.a.s.ArtifactoryConcurrentExecutor:71) - Task org.artifactory.concurrent.ArtifactoryRunnable@153c8638 was rejected by scheduler: Task org.artifactory.concurrent.ArtifactoryRunnable@153c8638 rejected from java.util.concurrent.ThreadPoolExecutor@75223fbf[Running, pool size = 16, active threads = 16, queued tasks = 10000, completed tasks = 183940]
2019-11-25 13:38:37,828 [http-nio-8081-exec-11] [ERROR] (o.a.s.a.AsyncAdvice :262) - Task org.artifactory.schedule.aop.CompoundInvocation@3a4eb471 rejected by scheduler: Executor [org.artifactory.schedule.ArtifactoryConcurrentExecutor@3ad754b4] did not accept task: org.artifactory.schedule.aop.AsyncAdvice$$Lambda$727/543397086@33521832
2019-11-25 13:38:37,830 [http-nio-8081-exec-11] [WARN ] (o.a.s.ArtifactoryConcurrentExecutor:71) - Task org.artifactory.concurrent.ArtifactoryRunnable@43cd88f9 was rejected by scheduler: Task org.artifactory.concurrent.ArtifactoryRunnable@43cd88f9 rejected from java.util.concurrent.ThreadPoolExecutor@75223fbf[Running, pool size = 16, active threads = 16, queued tasks = 10000, completed tasks = 183940]
2019-11-25 13:38:37,830 [http-nio-8081-exec-11] [ERROR] (o.a.r.c.e.m.GlobalExceptionMapper:48) - Executor [org.artifactory.schedule.ArtifactoryConcurrentExecutor@3ad754b4] did not accept task: org.artifactory.schedule.aop.AsyncAdvice$$Lambda$748/416148906@607d9b3a
org.springframework.core.task.TaskRejectedException: Executor [org.artifactory.schedule.ArtifactoryConcurrentExecutor@3ad754b4] did not accept task: org.artifactory.schedule.aop.AsyncAdvice$$Lambda$748/416148906@607d9b3a
{{ at org.springframework.core.task.support.TaskExecutorAdapter.submit(TaskExecutorAdapter.java:93)}}
{{ at org.springframework.scheduling.concurrent.ConcurrentTaskExecutor.submit(ConcurrentTaskExecutor.java:143)}}
{{ at org.artifactory.schedule.aop.AsyncAdvice.submitWorkQueueTask(AsyncAdvice.java:235)}}
{{ at org.artifactory.schedule.aop.AsyncAdvice.submit(AsyncAdvice.java:217)}}
{{ at org.artifactory.schedule.aop.AsyncAdvice.executeInvocation(AsyncAdvice.java:146)}}
{{ at org.artifactory.schedule.aop.AsyncAdvice.invoke(AsyncAdvice.java:124)}}
{{ at org.artifactory.schedule.aop.AsyncAdvice.invoke(AsyncAdvice.java:62)}}
{{ at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)}}
{{ at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207)}}
{{ at com.sun.proxy.$Proxy152.calculateMavenMetadataAsync(Unknown Source)}}
{{ at org.artifactory.repo.service.RepositoryServiceImpl.copyMultiTx(RepositoryServiceImpl.java:1104)}}
{{ at sun.reflect.GeneratedMethodAccessor1117.invoke(Unknown Source)}}
{{ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
{{ at java.lang.reflect.Method.invoke(Method.java:498)}}
{{ at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)}}
{{ at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201)}}
{{ at com.sun.proxy.$Proxy151.copyMultiTx(Unknown Source)}}
{{ at org.artifactory.addon.common.repomd.ArtifactoryService.copy(ArtifactoryService.java:453)}}
{{ at org.jfrog.repomd.docker.v2.rest.handler.DockerV2LocalRepoHandler.finishPatchUpload(DockerV2LocalRepoHandler.java:214)}}
{{ at org.jfrog.repomd.docker.v2.rest.handler.DockerV2LocalRepoHandler.uploadBlob(DockerV2LocalRepoHandler.java:192)}}
{{ at org.jfrog.repomd.docker.v2.rest.DockerV2Resource.uploadBlob(DockerV2Resource.java:71)}}
{{ at sun.reflect.GeneratedMethodAccessor1121.invoke(Unknown Source)}}
{{ at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)}}
{{ at java.lang.reflect.Method.invoke(Method.java:498)}}
{{ at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)}}
{{ at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)}}
{{ at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)}}
{{ at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302)}}
{{ at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)}}
{{ at com.sun.jersey.server.impl.uri.rules.SubLocatorRule.accept(SubLocatorRule.java:137)}}
{{ at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)}}
{{ at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)}}
{{ at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)}}
{{ at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)}}
{{ at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542)}}
{{ at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473)}}
{{ at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419)}}
{{ at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409)}}
{{ at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)}}
{{ at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)}}
{{ at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)}}
{{ at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)}}
{{ at org.artifactory.webapp.servlet.RepoFilter.execute(RepoFilter.java:186)}}
{{ at org.artifactory.webapp.servlet.RepoFilter.doFilter(RepoFilter.java:96)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)}}
{{ at org.artifactory.webapp.servlet.AccessFilter.useAuthentication(AccessFilter.java:426)}}
{{ at org.artifactory.webapp.servlet.AccessFilter.authenticateAndExecute(AccessFilter.java:305)}}
{{ at org.artifactory.webapp.servlet.AccessFilter.doFilterInternal(AccessFilter.java:208)}}
{{ at org.artifactory.webapp.servlet.AccessFilter.doFilter(AccessFilter.java:167)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)}}
{{ at org.artifactory.webapp.servlet.RequestFilter.doFilter(RequestFilter.java:77)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)}}
{{ at org.artifactory.webapp.servlet.ArtifactoryCsrfFilter.doFilter(ArtifactoryCsrfFilter.java:85)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)}}
{{ at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:164)}}
{{ at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:80)}}
{{ at org.artifactory.webapp.servlet.SessionFilter.doFilter(SessionFilter.java:62)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)}}
{{ at org.artifactory.webapp.servlet.ArtifactoryFilter.doFilter(ArtifactoryFilter.java:124)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)}}
{{ at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:728)}}
{{ at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:470)}}
{{ at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:395)}}
{{ at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:316)}}
{{ at org.artifactory.util.DockerInternalRewrite.redirect(DockerInternalRewrite.java:62)}}
{{ at org.artifactory.webapp.servlet.ArtifactoryFilter.redirectIfNeeded(ArtifactoryFilter.java:153)}}
{{ at org.artifactory.webapp.servlet.ArtifactoryFilter.doFilter(ArtifactoryFilter.java:109)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)}}
{{ at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)}}
{{ at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)}}
{{ at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)}}
{{ at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)}}
{{ at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:279)}}
{{ at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)}}
{{ at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)}}
{{ at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)}}
{{ at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)}}
{{ at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:800)}}
{{ at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)}}
{{ at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:800)}}
{{ at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1471)}}
{{ at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)}}
{{ at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)}}
{{ at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)}}
{{ at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)}}
{{ at java.lang.Thread.run(Thread.java:748)}}
Caused by: java.util.concurrent.RejectedExecutionException: Task org.artifactory.concurrent.ArtifactoryRunnable@43cd88f9 rejected from java.util.concurrent.ThreadPoolExecutor@75223fbf[Running, pool size = 16, active threads = 16, queued tasks = 10000, completed tasks = 183940]
{{ at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)}}
{{ at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)}}
{{ at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)}}
{{ at org.artifactory.schedule.ArtifactoryConcurrentExecutor.execute(ArtifactoryConcurrentExecutor.java:69)}}
{{ at org.springframework.core.task.support.TaskExecutorAdapter.submit(TaskExecutorAdapter.java:88)}}
{{ ... 93 common frames omitted}}