Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-17311

Document that running background jobs can cause a system restore to fail

    Details

    • Type: Documentation
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: 6.1.0
    • Fix Version/s: None
    • Component/s: Backup
    • Labels:
      None

      Description

      This is a follow-up to RTFACT-16782 and to the support ticket #82959:

      When performing a system restore while background jobs are running, the system restore can fail because locks are held for too long. This results in a stacktrace (redacted) like the following in the import log:

      2018-05-27 10:57:32,769 [art-import-2] [ERROR] (o.a.r.d.i.DbRepoImportHandler:376) - Import error: from: /backup/artifactory/current/repositories/somerepo/archives/2017/someartifact2017_archive0.zip to somerepo:archives/2017/someartifact2017_archive0.zip reason: Could not import file '/backup/artifactory/current/repositories/somerepo/archives/2017/someartifact2017_archive0.zip into somerepo:archives/2017/someartifact2017_archive0.zip.
       org.artifactory.concurrent.LockingException: Lock on LockEntryId somerepo:archives/2017/someartifact2017_archive0.zip not acquired in 120 seconds. Lock info: org.artifactory.storage.fs.lock.provider.JVMLockWrapper@5fe5ca9d.
       at org.artifactory.storage.fs.lock.SessionLockEntry.acquire(SessionLockEntry.java:105)
       at org.artifactory.storage.fs.lock.SessionLockEntry.acquireWriteLock(SessionLockEntry.java:90)
       at org.artifactory.storage.db.fs.session.SqlStorageSession.writeLock(SqlStorageSession.java:88)
       at org.artifactory.storage.fs.lock.LockingHelper.writeLock(LockingHelper.java:55)
       at org.artifactory.storage.db.fs.model.DbFsItemProvider.getOrCreateMutableFsItem(DbFsItemProvider.java:147)
       at org.artifactory.storage.db.fs.model.DbMutableFileProvider.getOrCreMutableFile(DbMutableFileProvider.java:65)
       at org.artifactory.repo.db.DbStoringRepoMixin.createOrGetFile(DbStoringRepoMixin.java:670)
       at org.artifactory.repo.db.DbLocalRepo.createOrGetFile(DbLocalRepo.java:247)
       at org.artifactory.repo.db.importexport.DbRepoImportHandler.deployImportedFile(DbRepoImportHandler.java:345)
       at org.artifactory.repo.db.importexport.DbRepoImportHandler.importFile(DbRepoImportHandler.java:318)
       at org.artifactory.repo.db.importexport.DbRepoImportHandler.importRecursive(DbRepoImportHandler.java:259)
       at org.artifactory.repo.db.importexport.DbRepoImportHandler.importRecursive(DbRepoImportHandler.java:271)
       at org.artifactory.repo.db.importexport.DbRepoImportHandler.importRecursive(DbRepoImportHandler.java:271)
       at org.artifactory.repo.db.importexport.DbRepoImportHandler.importRecursive(DbRepoImportHandler.java:271)
       at org.artifactory.repo.db.importexport.DbRepoImportHandler.executeRecursiveImport(DbRepoImportHandler.java:191)
       at org.artifactory.repo.db.importexport.DbRepoImportHandler.executeImport(DbRepoImportHandler.java:156)
       at org.artifactory.repo.service.ImportHandlerCallablesFactory.lambda$create$0(ImportHandlerCallablesFactory.java:58)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at org.artifactory.concurrent.ArtifactoryRunnable.run(ArtifactoryRunnable.java:30)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)
       2018-05-27 10:58:34,778 [art-import-2] [ERROR] (o.a.r.d.i.DbRepoImportHandler:376) - Import error: from: /backup/artifactory/current/repositories/somerepo/archives/2017/someartifact2017_archive14.zip to somerepo:archives/2017/someartifact2017_archive14.zip reason: Could not import file '/backup/artifactory/current/repositories/somerepo/archives/2017/someartifact2017_archive14.zip into somerepo:archives/2017/someartifact2017_archive14.zip.: Lock on LockEntryId somerepo:archives not acquired in 120 seconds. Lock info: org.artifactory.storage.fs.lock.provider.JVMLockWrapper@64e15163.

      From the stacktrace and the documentation it's unclear what the problem could be and that running background jobs could be the culprit. This is made worse by the fact that the Artifactory UI does not display that background jobs are currently being run and that they might cause a system restore to fail.

      Therefore I suggest that the documentation is amended with:

      • a warning, that running background jobs can cause the system restore to fail if locks are being held for too long (e.g. because the system I/O is not fast enough) which results in LockingException and friends.
      • a mention of the possibility to check the state of the background jobs via the REST API

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                aahlenst Andreas Ahlenstorf
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated: