-
Type:
Bug
-
Status: Done
-
Resolution: Done
-
Affects Version/s: None
-
Fix Version/s: 7.35.1
-
Component/s: None
-
Labels:None
-
Environment:
DB type & Version - Derby
Is this an HA env? Single node
Is this On-Prem or SaaS? On-Prem
-
Severity:Critical
-
Location:External
Problem description:
When attempting to resolve from cargo repositories a BufferOverflowException can occur.
In the Artifactory logs, it will be trying to read a file from the internal .git folder and get a BufferOverflowException.
[jfrt ] [ERROR] [15bf295bbdb4b95b] [.ArtifactoryReadableChannel:73] [-nio-8081-exec-12689] Error reading file: <REPO_NAME>:.git/pack-687812346-INSERT.ref java.nio.BufferOverflowException: null at java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:221) at java.base/java.nio.ByteBuffer.put(ByteBuffer.java:914) at org.artifactory.addon.git.service.ArtifactoryReadableChannel.read(ArtifactoryReadableChannel.java:69) at org.eclipse.jgit.internal.storage.dfs.BlockBasedFile.read(BlockBasedFile.java:169) at org.eclipse.jgit.internal.storage.dfs.BlockBasedFile.readOneBlock(BlockBasedFile.java:139) at org.eclipse.jgit.internal.storage.dfs.DfsBlockCache.getOrLoad(DfsBlockCache.java:382) at org.eclipse.jgit.internal.storage.dfs.DfsReftable$CacheSource.read(DfsReftable.java:98) at org.eclipse.jgit.internal.storage.reftable.ReftableReader.readHeaderOrFooter(ReftableReader.java:338) at org.eclipse.jgit.internal.storage.reftable.ReftableReader.readFileHeader(ReftableReader.java:287) at org.eclipse.jgit.internal.storage.reftable.ReftableReader.allRefs(ReftableReader.java:134) at org.eclipse.jgit.internal.storage.reftable.MergedReftable.allRefs(MergedReftable.java:109) at org.eclipse.jgit.internal.storage.reftable.ReftableDatabase.getRefsByPrefix(ReftableDatabase.java:254) at org.eclipse.jgit.internal.storage.dfs.DfsReftableDatabase.getRefsByPrefix(DfsReftableDatabase.java:176) at org.eclipse.jgit.lib.RefDatabase.getRefs(RefDatabase.java:361) at org.eclipse.jgit.transport.UploadPack.getAdvertisedOrDefaultRefs(UploadPack.java:877) at org.eclipse.jgit.transport.UploadPack.sendAdvertisedRefs(UploadPack.java:1503) at org.eclipse.jgit.http.server.UploadPackServlet$InfoRefs.respond(UploadPackServlet.java:102) at org.eclipse.jgit.http.server.SmartServiceInfoRefs.service(SmartServiceInfoRefs.java:103) at org.eclipse.jgit.http.server.SmartServiceInfoRefs.access$1(SmartServiceInfoRefs.java:94) at org.eclipse.jgit.http.server.SmartServiceInfoRefs$Chain.doFilter(SmartServiceInfoRefs.java:186) at org.artifactory.addon.git.filter.GitRepositoryUpdateFilter.doFilter(GitRepositoryUpdateFilter.java:54) at org.eclipse.jgit.http.server.SmartServiceInfoRefs$Chain.doFilter(SmartServiceInfoRefs.java:184) at org.eclipse.jgit.http.server.SmartServiceInfoRefs.doFilter(SmartServiceInfoRefs.java:85) at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:209) at org.eclipse.jgit.http.server.RepositoryFilter.doFilter(RepositoryFilter.java:112) at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:209) at org.eclipse.jgit.http.server.NoCacheFilter.doFilter(NoCacheFilter.java:53) at org.eclipse.jgit.http.server.glue.UrlPipeline$Chain.doFilter(UrlPipeline.java:209) at org.eclipse.jgit.http.server.glue.UrlPipeline.service(UrlPipeline.java:188) at org.eclipse.jgit.http.server.glue.SuffixPipeline.service(SuffixPipeline.java:70) at org.eclipse.jgit.http.server.glue.MetaFilter.doFilter(MetaFilter.java:150) at org.eclipse.jgit.http.server.glue.MetaServlet.service(MetaServlet.java:109) at javax.servlet.http.HttpServlet.service(HttpServlet.java:733) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.RepoFilter.execute(RepoFilter.java:203) at org.artifactory.webapp.servlet.RepoFilter.doFilter(RepoFilter.java:105) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.AccessFilter.useAuthentication(AccessFilter.java:563) at org.artifactory.webapp.servlet.AccessFilter.useAnonymousIfPossible(AccessFilter.java:522) at org.artifactory.webapp.servlet.AccessFilter.doFilterInternal(AccessFilter.java:275) at org.artifactory.webapp.servlet.AccessFilter.doFilter(AccessFilter.java:207) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.RequestFilter.doFilter(RequestFilter.java:87) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.ArtifactoryCsrfFilter.doFilter(ArtifactoryCsrfFilter.java:83) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.springframework.session.web.http.SessionRepositoryFilter.doFilterInternal(SessionRepositoryFilter.java:164) at org.springframework.session.web.http.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:80) at org.artifactory.webapp.servlet.SessionFilter.doFilter(SessionFilter.java:67) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.ArtifactoryTracingFilter.doFilter(ArtifactoryTracingFilter.java:32) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.artifactory.webapp.servlet.ArtifactoryFilter.doFilter(ArtifactoryFilter.java:123) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:201) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:544) at org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:305) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:364) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:616) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:831) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1629) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.base/java.lang.Thread.run(Thread.java:834)
In the Artifactory access logs you can see a denied download:
[DENIED DOWNLOAD] <REPO_NAME>:.git/pack-687812346-INSERT.ref for client : NA / 127.0.0.1.
What is the impact on the customer?
This prevents resolving from the Cargo repository.
What is the expected behavior?
Artifactory should not encounter a BufferOverflowException
Reproduction Steps(2 sets):
Reproduction Steps 1:
1. Create a Cargo local repository. (I have named mine cargo-local)
2. Configure your config.toml accordingly.
3. Run cargo commands to generate a crate:
cargo new sample_package; cd sample_package; cargo build
4. Publish the crate to Artifactory many times.
Here is a sample loop that achieves this. (This loop publishes then removes the crate in Artifactory. Note replace the variables for your environment).
for i in {1..200}; do cargo publish --token "Bearer <ACCESS_TOKEN>" --allow-dirty ; curl -u<USER>:<PASSWORD> -XDELETE "<ART_URL>artifactory/cargo-local/crates/<PATH_TO_FILE>"; done
Observe that eventually, you will start getting a "java.nio.BufferOverflowException: null" error in the logs. Other Cargo commands like cargo install.. will also not work because of the overflow errors.
You can also observe that these errors start happening at around 400 entries in the .git folder of the repository.
You can check with the following curl command:
curl -uadmin:password "<ART_URL>/artifactory/cargo-local/.git" -vL | grep pack | wc -l
Reproduction Steps 2:
- Create a default Cargo remote repository and Cargo local repository
- In the ~/.cargo/config.toml configure the index and source
[registry] default = "artifactory" [registries.artifactory] index = "<ART_URL>/artifactory/git/<CARGO_REPO>.git" [net] git-fetch-with-cli = true [source.artifactory] registry = "<ART_URL>/artifactory/git/<CARGO_REPO>.git" [source.crates-io] replace-with = "artifactory"
3. Create a simple Cargo project i.e.
cargo new hello_world
4. In the project add a bunch of dependencies:
[dependencies] time = "0.1.12" regex = "0.1.41" rand = "0.8.4" syn = "1.0.81" rand_core = "0.6.3" libc = "0.2.107" quote = "1.0.10" cfg-if = "1.0.0" proc-macro2 = "1.0.32" bitflags = "1.3.2" enum_dispatch = "0.3.7" cargo-audit = "0.16.0" honggfuzz = "0.5.54" cargo-readme = "3.2.0" cargo-hack = "0.5.8" cargo-deb = "1.34.0" cargo-make = "0.35.6" cargo-chef = "0.1.32" cargo-deny = "0.10.1" cargo-edit = "0.8.0"
5. Run 'cargo build'. This will populate the cargo remote repository.
6. Copy the cargo remote cache to the cargo local repository.
7. Change the config.toml so that the source is pointing to the local repository:
[source.artifactory]
registry = "<ART_URL>/artifactory/git/cargo-local.git"
7. Remove the local cargo index and cache from where you are running the cargo commands
rm -r ~/.cargo/registry/cache/* rm -r ~/.cargo/registry/index/*
8. Perform 'cargo fmt --all – --check --verbose' and observe the overflow in Artifactory service logs.
