Uploaded image for project: 'Jenkins Artifactory Plug-in'
  1. Jenkins Artifactory Plug-in
  2. HAP-983

Pipeline download of large artifact results in java.lang.OutOfMemoryError

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: 2.12.1
    • Fix Version/s: 2.13.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Jenkins Master: Jenkins 2.77 running on 64-bit Linux
      Affected Jenkins agents: Windows 10 64-bit, running latest agent version
      Artifactory server version: 5.1.4 rev 50021

      Description

      When downloading an artifact with a size of 582.07 MB to a Windows agent within a pipeline job, a java.lang.OutOfMemoryError is thrown with the following stack trace:

      java.lang.OutOfMemoryError: Java heap space
      	at org.apache.commons.io.output.ByteArrayOutputStream.needNewBuffer(ByteArrayOutputStream.java:122)
      	at org.apache.commons.io.output.ByteArrayOutputStream.write(ByteArrayOutputStream.java:196)
      	at org.apache.commons.io.output.ByteArrayOutputStream.toBufferedInputStream(ByteArrayOutputStream.java:279)
      	at org.apache.commons.io.IOUtils.toBufferedInputStream(IOUtils.java:431)
      	at org.jfrog.build.client.PreemptiveHttpClient$PreemptiveHttpClientHandler.handleResponse(PreemptiveHttpClient.java:318)
      	at org.jfrog.build.client.PreemptiveHttpClient$PreemptiveHttpClientHandler.handleResponse(PreemptiveHttpClient.java:309)
      	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:222)
      	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:164)
      	at org.jfrog.build.client.PreemptiveHttpClient.execute(PreemptiveHttpClient.java:117)
      	at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryDependenciesClient.execute(ArtifactoryDependenciesClient.java:185)
      	at org.jfrog.build.extractor.clientConfiguration.client.ArtifactoryDependenciesClient.downloadArtifact(ArtifactoryDependenciesClient.java:166)
      	at org.jfrog.build.extractor.clientConfiguration.util.DependenciesDownloaderHelper.downloadArtifact(DependenciesDownloaderHelper.java:227)
      	at org.jfrog.build.extractor.clientConfiguration.util.DependenciesDownloaderHelper.downloadDependencies(DependenciesDownloaderHelper.java:115)
      	at org.jfrog.build.extractor.clientConfiguration.util.DependenciesDownloaderImpl.download(DependenciesDownloaderImpl.java:48)
      	at org.jfrog.build.extractor.clientConfiguration.util.AqlDependenciesHelper.downloadDependencies(AqlDependenciesHelper.java:46)
      	at org.jfrog.build.extractor.clientConfiguration.util.WildcardsDependenciesHelper.retrievePublishedDependencies(WildcardsDependenciesHelper.java:88)
      	at org.jfrog.build.extractor.clientConfiguration.util.DependenciesDownloaderHelper.downloadDependencies(DependenciesDownloaderHelper.java:83)
      	at org.jfrog.build.extractor.clientConfiguration.util.spec.SpecsHelper.downloadArtifactsBySpec(SpecsHelper.java:96)
      	at org.jfrog.hudson.generic.FilesResolverCallable.invoke(FilesResolverCallable.java:48)
      	at org.jfrog.hudson.generic.FilesResolverCallable.invoke(FilesResolverCallable.java:20)
      	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:2750)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:181)
      	at hudson.remoting.UserRequest.perform(UserRequest.java:52)
      	at hudson.remoting.Request$2.run(Request.java:336)
      	at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
      	at java.util.concurrent.FutureTask.run(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
      	at hudson.remoting.Engine$1$1.run(Engine.java:98)
      	at java.lang.Thread.run(Unknown Source)
      	at ......remote call to JNLP4-connect connection from 192.168.9.77/192.168.9.77:55006(Native Method)
      	at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1647)
      	at hudson.remoting.UserResponse.retrieve(UserRequest.java:308)
      	at hudson.remoting.Channel.call(Channel.java:896)
      Caused: java.io.IOException: Remote call on JNLP4-connect connection from 192.168.9.77/192.168.9.77:55006 failed
      	at hudson.remoting.Channel.call(Channel.java:904)
      	at hudson.FilePath.act(FilePath.java:987)
      Caused: java.io.IOException: remote file operation failed: C:\jenkins\workspace\HPS\MASTER\shared at hudson.remoting.Channel@166238f1:JNLP4-connect connection from 192.168.9.77/192.168.9.77:55006
      	at hudson.FilePath.act(FilePath.java:994)
      	at hudson.FilePath.act(FilePath.java:976)
      	at org.jfrog.hudson.pipeline.executors.GenericDownloadExecutor.execution(GenericDownloadExecutor.java:41)
      	at org.jfrog.hudson.pipeline.steps.DownloadStep$Execution.run(DownloadStep.java:66)
      	at org.jfrog.hudson.pipeline.steps.DownloadStep$Execution.run(DownloadStep.java:46)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
      	at hudson.security.ACL.impersonate(ACL.java:260)
      	at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

      Launching the agent on the node machine with a larger Java heap size works around the problem, but it seems wrong that (I'm presuming) the entire file is being stored in memory before serializing it to disk. Our node configuration would be simplified without having to work around this issue.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                eyalb Eyal Ben Moshe
                Reporter:
                trask Trask Rogers
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: