[HAP-937] Support JFrog DSL from within Docker containers Created: 09/Jun/17  Updated: 14/Nov/17  Resolved: 27/Sep/17

Status: Resolved
Project: Jenkins Artifactory Plug-in
Component/s: Gradle, Maven3
Affects Version/s: 2.12.2
Fix Version/s: 2.13.0

Type: Improvement Priority: Normal
Reporter: David Liederman Assignee: Yahav Itzhak
Resolution: Fixed Votes: 4
Labels: None
Environment:

Jenkins 2.32.3, Artifactory plugin 2.11.0, Pipeline 2.5



 Description   

How to run Jenkins pipeline jobs using JFrog DSL from within Docker containers?
def buildenv = docker.image('container')
buildenv.pull()
buildenv.inside {
def server = Artifactory.server('OLP Staging Artifactory')
def buildInfo = Artifactory.newBuildInfo()
buildInfo.env.capture = true

stage('Build') {
timeout(10)

{ def rtGradle = Artifactory.newGradleBuild() rtGradle.useWrapper = true rtGradle.resolver repo:'remote-repos', server: server rtGradle.run tasks: 'test jacocoTestReport' }

}
}
will fail with: Could not read initialization script '/tmp/init-artifactory6638392993045452201gradle' as it does not exist.

same code running on the plain slave will pass thru correctly. Docker containers are widely used in CD flows running on top of stateless nodes in the cloud.



 Comments   
Comment by Stefan Lengauer [ 05/Jul/17 ]

I have experienced the same Issue a while ago and submitted an Issue at Jenkins https://issues.jenkins-ci.org/browse/JENKINS-40035

A possible workaround is mounting /tmp from the slave into the container.

Comment by Waldek Maleska [ 05/Jul/17 ]

I've been trying to work around the issue by mounting the tmp directory as `-v /tmp:/tmp:rw` however that didn't really help;
The build started fine but at the end, running within a container fails at savint the buildInfo.

```
12 actionable tasks: 12 executed
ERROR: Couldn't read generated build info at : /tmp/generated.build.info6736242178482504438.json
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
$ docker stop --time=1 e66dca1e0a8076647b7cf6c0c4b311137326e12dd3c97a0fd807e1f93b37e310
$ docker rm -f e66dca1e0a8076647b7cf6c0c4b311137326e12dd3c97a0fd807e1f93b37e310
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
hudson.model.Run$RunnerAbortedException
at org.jfrog.hudson.pipeline.Utils.getGeneratedBuildInfo(Utils.java:190)
at org.jfrog.hudson.pipeline.steps.ArtifactoryGradleBuild$Execution.run(ArtifactoryGradleBuild.java:133)
at org.jfrog.hudson.pipeline.steps.ArtifactoryGradleBuild$Execution.run(ArtifactoryGradleBuild.java:102)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1$1.call(AbstractSynchronousNonBlockingStepExecution.java:47)
at hudson.security.ACL.impersonate(ACL.java:221)
at org.jenkinsci.plugins.workflow.steps.AbstractSynchronousNonBlockingStepExecution$1.run(AbstractSynchronousNonBlockingStepExecution.java:44)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
Finished: FAILURE
```

Even though the container mounts /tmp, the `generated.build` is either empty or isn't created (on the host) at all.

Comment by Stefan Lengauer [ 05/Jul/17 ]

I do have a gradle build inside my container und it works with the mounted /tmp :

docker.image(container.imageName()).inside('-v /tmp:/tmp:rw') {
              def gradleBuildInfo = Artifactory.newBuildInfo()
              def rtGradle = Artifactory.newGradleBuild()
              rtGradle.setUseWrapper(true)
              rtGradle.setUsesPlugin(true)
              rtGradle.resolver server: artifactory, repo: 'dependencies'              
              rtGradle.run buildFile: './build.gradle', tasks: 'compileGroovy', buildInfo: gradleBuildInfo, switches: '--no-daemon'
              buildInfo.append(gradleBuildInfo)
              buildInfo.env.collect()
              artifactory.publishBuildInfo(buildInfo)
}

This is basically the code that is working in my environment without issues. You have to be in the same node{} block as the container for this to work.

Comment by Waldek Maleska [ 20/Jul/17 ]

Yes, I've finally managed to replicate the setup with mounting /tmp and indeed that workaround worked. Thank you, Stefan.

This is as long as you set `rtGradle.setUsesPlugin(true)`

When you try to *not* have any Artifactory setup within the Gradle build files, this fails as jfrog classes get installed on the slave (outside of Docker container) but are expected to be available within the container.

2017-07-20T18:10:35.700+0100 Creating new cache for plugin-use-metadata, path /home/jenkins/.gradle/caches/4.0/plugin-resolution/plugin-use-metadata.bin, access org.gradle.cache.internal.DefaultCacheAccess@544630b7
2017-07-20T18:10:35.702+0100 Creating new cache for client-status, path /home/jenkins/.gradle/caches/4.0/plugin-resolution/client-status.bin, access org.gradle.cache.internal.DefaultCacheAccess@544630b7
2017-07-20T18:10:35.844+0100 Starting Build
2017-07-20T18:10:35.943+0100 Compiling initialization script '/tmp/init-artifactory3567342617109570097gradle' using SubsetScriptTransformer.
2017-07-20T18:10:36.744+0100 Creating new cache for metadata-1.1/results, path /home/jenkins/.gradle/caches/transforms-1/metadata-1.1/results.bin, access org.gradle.cache.internal.DefaultCacheAccess@ad3324b
2017-07-20T18:10:36.982+0100 file or directory '/var/spool/jenkins/cache/artifactory-plugin/2.12.1', not found
2017-07-20T18:10:36.986+0100 Compiling initialization script '/tmp/init-artifactory3567342617109570097gradle' using BuildScriptTransformer.
2017-07-20T18:10:37.020+0100 
2017-07-20T18:10:37.021+0100 FAILURE: Build failed with an exception.
2017-07-20T18:10:37.021+0100 
2017-07-20T18:10:37.021+0100 * Where:
2017-07-20T18:10:37.021+0100 Initialization script '/tmp/init-artifactory3567342617109570097gradle' line: 1
2017-07-20T18:10:37.022+0100 
2017-07-20T18:10:37.023+0100 * What went wrong:
2017-07-20T18:10:37.023+0100 Could not compile initialization script '/tmp/init-artifactory3567342617109570097gradle'.
2017-07-20T18:10:37.023+0100 > startup failed:
2017-07-20T18:10:37.023+0100   initialization script '/tmp/init-artifactory3567342617109570097gradle': 1: unable to resolve class org.jfrog.gradle.plugin.artifactory.ArtifactoryPlugin
2017-07-20T18:10:37.023+0100    @ line 1, column 1.
2017-07-20T18:10:37.024+0100      import org.jfrog.gradle.plugin.artifactory.ArtifactoryPlugin
2017-07-20T18:10:37.024+0100      ^
2017-07-20T18:10:37.024+0100   
2017-07-20T18:10:37.024+0100   initialization script '/tmp/init-artifactory3567342617109570097gradle': 2: unable to resolve class org.jfrog.gradle.plugin.artifactory.task.ArtifactoryTask
2017-07-20T18:10:37.025+0100    @ line 2, column 1.
2017-07-20T18:10:37.025+0100      import org.jfrog.gradle.plugin.artifactory.task.ArtifactoryTask
2017-07-20T18:10:37.025+0100      ^
2017-07-20T18:10:37.025+0100   
2017-07-20T18:10:37.026+0100   2 errors
2017-07-20T18:10:37.026+0100 
2017-07-20T18:10:37.026+0100 
2017-07-20T18:10:37.026+0100 * Try:
2017-07-20T18:10:37.026+0100 Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output.
2017-07-20T18:10:37.027+0100 
2017-07-20T18:10:37.027+0100 BUILD FAILED in 9s

One could again try to work around that using:

-v /var/spool/jenkins:/var/spool/jenkins:ro

but that's as fishy as it gets.

Comment by Yahav Itzhak [ 10/Sep/17 ]

David Liederman,
We fixed the issue. Here is the link to the commit:
https://github.com/JFrogDev/jenkins-artifactory-plugin/commit/0f7de88a11b778691db481641cf7dfc04b50a5fe
Here is a download link for a snapshot version which includes the fix:
https://oss.jfrog.org/oss-snapshot-local/org/jenkins-ci/plugins/artifactory/2.12.3-SNAPSHOT/artifactory-2.12.3-20170910.063909-13.hpi
We would appreciate your feedback for it.

Comment by David Liederman [ 11/Sep/17 ]

Do you expect me to test the snapshot or it would be enough to provide a feedback on the next plugin release? Right now I don't have an environment where I can push a snapshot plugin so it would be much easier to verify the next released plugin fixes the problem. I'll have to recreate the job configuration - meanwhile I abandoned it and used native environments instead of containers. But I'm very interested to migrate my builds into containerized environments ASAP so I'll try to reproduce the configuration as soon as I see the new release of the plugin and provide the feedback here.

Comment by Yahav Itzhak [ 27/Sep/17 ]

David Liederman,
Just letting you know that version 2.13.0 is released and includes this fix.

Comment by David Liederman [ 29/Sep/17 ]

Thanks.
Can you provide a link to an example of a functional Jenkinsfile?

Comment by Yahav Itzhak [ 02/Oct/17 ]

Hi David Liederman,
We uploaded an example. You can find fine it here.

Generated at Thu May 23 03:38:52 UTC 2019 using JIRA 7.6.3#76005-sha1:8a4e38d34af948780dbf52044e7aafb13a7cae58.