-
Type:
Bug
-
Status: Open
-
Priority:
Critical
-
Resolution: Unresolved
-
Affects Version/s: 6.8.7
-
Fix Version/s: None
-
Component/s: Access Server, Import/export
-
Labels:
-
Environment:
Just a very large permissions export file for importing.
We saw a failure on a 24GB of JVM Heap when importing ~2GB json file.
When importing very large security entities export-files, Access tries to read the entire file into memory. This is not scalable and fails with OOME for large files (in our example ~2GB).
Example stack trace:
Caused by: java.lang.OutOfMemoryError: null
at java.lang.AbstractStringBuilder.hugeCapacity(AbstractStringBuilder.java:161)
at java.lang.AbstractStringBuilder.newCapacity(AbstractStringBuilder.java:155)
at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:125)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:596)
at java.lang.StringBuilder.append(StringBuilder.java:190)
at org.apache.commons.io.output.StringBuilderWriter.write(StringBuilderWriter.java:142)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2538)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:2516)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2493)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:2441)
at org.apache.commons.io.IOUtils.toString(IOUtils.java:1084)
at org.apache.commons.io.FileUtils.readFileToString(FileUtils.java:1806)
at org.jfrog.access.server.service.backup.AccessImporterExporterImpl.doImport(AccessImporterExporterImpl.java:74)