-
Type:
Improvement
-
Status: Done
-
Resolution: Cannot Reproduce
-
Affects Version/s: 5.10.2
-
Fix Version/s: None
-
Component/s: Kubernetes, RPM, YUM
-
Labels:
With the growing popularity of Kubernetes, customers want to proxy the Kubernetes components repository:
https://packages.cloud.google.com
However, Google is using relative paths, for example: (note the ../)
Artifactory cannot handle this kind of path and returns the following error:
java.nio.file.InvalidPathException: Path element cannot end with a dot: google-packages-cache/yum/repos/kubernetes-el7-x86_64/../.
Steps to reproduce:
- Create an RPM remote repository and point it to: https://packages.cloud.google.com/
- Use the following baseurl: baseurl=http://<IP>:8081/artifactory/rpm-remote-google/yum/repos/kubernetes-el7-x86_64/
- Run "yum install kubectl"
Full stacktrace:
2018-04-18 11:48:48,312 [http-nio-8081-exec-8] [ERROR] (o.a.r.d.DbStoringRepoMixin:274) - Couldn't save resource, reason: java.nio.file.InvalidPathException: Path element cannot end with a dot: rpm-remote-google-cache/yum/repos/kubernetes-el7-x86_64/../ at org.artifactory.util.PathValidator.validate(PathValidator.java:74) at org.artifactory.storage.db.fs.model.DbMutableItem.<init>(DbMutableItem.java:109) at org.artifactory.storage.db.fs.model.DbMutableFolder.<init>(DbMutableFolder.java:46) at org.artifactory.storage.db.fs.model.DbFsItemProvider.createAncestors(DbFsItemProvider.java:215) at org.artifactory.storage.db.fs.model.DbFsItemProvider.getOrCreateMutableFsItem(DbFsItemProvider.java:144) at org.artifactory.storage.db.fs.model.DbMutableFileProvider.getOrCreMutableFile(DbMutableFileProvider.java:65) at org.artifactory.repo.db.DbStoringRepoMixin.createOrGetFile(DbStoringRepoMixin.java:670) at org.artifactory.repo.db.DbStoringRepoMixin.saveResource(DbStoringRepoMixin.java:187) at org.artifactory.repo.db.DbLocalRepo.saveResource(DbLocalRepo.java:154) at org.artifactory.repo.service.RepositoryServiceImpl.saveResourceInTransaction(RepositoryServiceImpl.java:1821) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:281) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) at com.sun.proxy.$Proxy149.saveResourceInTransaction(Unknown Source) at org.artifactory.repo.service.RepositoryServiceImpl.saveResource(RepositoryServiceImpl.java:1791) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) at com.sun.proxy.$Proxy149.saveResource(Unknown Source) at org.artifactory.repo.RemoteRepoBase.doDownloadAndSave(RemoteRepoBase.java:751) at org.artifactory.repo.RemoteRepoBase.downloadAndSave(RemoteRepoBase.java:594) at org.artifactory.repo.service.RepositoryServiceImpl.downloadAndSave(RepositoryServiceImpl.java:1719) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) at com.sun.proxy.$Proxy149.downloadAndSave(Unknown Source) at org.artifactory.repo.RemoteRepoBase.getResourceStreamHandle(RemoteRepoBase.java:498) at org.artifactory.repo.service.RepositoryServiceImpl.getResourceStreamHandle(RepositoryServiceImpl.java:1765) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:201) at com.sun.proxy.$Proxy149.getResourceStreamHandle(Unknown Source) at org.artifactory.engine.DownloadServiceImpl.respondFoundResource(DownloadServiceImpl.java:325) at org.artifactory.engine.DownloadServiceImpl.respond(DownloadServiceImpl.java:271) at org.artifactory.engine.DownloadServiceImpl.process(DownloadServiceImpl.java:205) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.artifactory.request.aop.RequestAdvice.invoke(RequestAdvice.java:67) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:207) at com.sun.proxy.$Proxy185.process(Unknown Source) at org.artifactory.webapp.servlet.RepoFilter.doDownload(RepoFilter.java:237) at org.artifactory.webapp.servlet.RepoFilter.execute(RepoFilter.java:159) at org.artifactory.webapp.servlet.RepoFilter.doFilter(RepoFilter.java:93) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.authentication.ArtifactoryAuthenticationFilterChain.lambda$doFilter$1(ArtifactoryAuthenticationFilterChain.java:133) at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:201) at org.artifactory.webapp.servlet.authentication.ArtifactoryBasicAuthenticationFilter.doFilter(ArtifactoryBasicAuthenticationFilter.java:84) at org.artifactory.webapp.servlet.authentication.ArtifactoryAuthenticationFilterChain.doFilter(ArtifactoryAuthenticationFilterChain.java:169) at org.artifactory.webapp.servlet.AccessFilter.authenticateAndExecute(AccessFilter.java:311) at org.artifactory.webapp.servlet.AccessFilter.doFilterInternal(AccessFilter.java:208) at org.artifactory.webapp.servlet.AccessFilter.doFilter(AccessFilter.java:167) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.RequestFilter.doFilter(RequestFilter.java:74) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:164) at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:80) at org.artifactory.webapp.servlet.SessionFilter.doFilter(SessionFilter.java:62) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.ArtifactoryFilter.doFilter(ArtifactoryFilter.java:128) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:279) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) 2018-04-18 11:48:48,320 [http-nio-8081-exec-8] [ERROR] (o.a.r.HttpRepo :1214) - rpm-remote-google: Failed to download 'https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/../../pool/b8f0cc3bc85e8614f0340547b14ca5377410afd087f51077410945a594f0b71b-kubectl-1.10.1-0.x86_64.rpm'. Received status code 200 and caught exception: Path element cannot end with a dot: rpm-remote-google-cache/yum/repos/kubernetes-el7-x86_64/../ 2018-04-18 11:48:48,322 [http-nio-8081-exec-8] [WARN ] (o.a.r.ArtifactoryResponseBase:107) - Sending HTTP error code 500: Could not process download request: Path element cannot end with a dot: rpm-remote-google-cache/yum/repos/kubernetes-el7-x86_64/../
Workarounds:
- Disable "Store Artifacts Locally"
or
2. Install the following user-plugin that converts the path to a supported one by omitting a folder from the path for each instance of: ../:
import org.artifactory.repo.RepoPath import org.artifactory.repo.RepoPathFactory import org.artifactory.request.Request download { beforeDownloadRequest { Request request, RepoPath path -> if (path.getRepoKey().equals("rpm-remote-google") && (path.toString().contains(".."))){ // path cannot be null log.info("Detected original request path which contains double dots in it: " + path.toString() + ". Trying to fix it...") try { def tmpPath = path.getPath() while (tmpPath.lastIndexOf("../")!=-1){ start = tmpPath.substring(0,tmpPath.indexOf("../")-1) end = tmpPath.substring(tmpPath.indexOf("../")+3) start = start.substring(0,start.lastIndexOf("/")+1) tmpPath = start+end } log.info("The original path: " + path.getPath() + " was modified to: " + tmpPath) modifiedRepoPath = RepoPathFactory.create(path.repoKey, tmpPath) } catch (Exception e){ log.warn("Unable to override path: " + e.getMessage()) } } } }
Note that the plugin has the repository key hard-coded (currently "rpm-remote-google").
Artifactory should support this kind of repositories in a more native way. This Jira is for packages.cloud.google.com but it could happen for different repositories as well.
- duplicates
-
RTFACT-17165 packagecloud rabbitmq-server failure
- Done
- is duplicated by
-
RTFACT-15200 Allow Kubernetes remote YUM repo
- Done
-
RTFACT-19135 HTTP 500 for yum install involving relative URLs
- Done
- is related to
-
RTFACT-19041 mvn site deploy to Artifactory with dav protocol results in a failure due to the presence of dot dot ".." as path element
- Open
- relates to
-
RTFACT-9141 RPM remote repository feature does not work with certain repositories
- Done