Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-19795

Build Promotion fails after upgrade from 6.9.0 to 6.11.1

    Details

      Description

      Build promotion from Jenkins with build ID XX (for example:23) to Artifactory fails with "HTTP Error 500: `Could not delete build 0 " error. Build 0 doesn't exists in Artifactory and user already had delete permissions for the build 23, set in a permission target.

      Error seen by enabling below Artifactory logger:
      <logger name="org.artifactory.build.BuildServiceImpl">
      <level value="debug" />  
      </logger>
       

       

      2019-07-24 21:36:48,033 [http-nio-12000-exec-19] [DEBUG] (o.a.b.BuildServiceImpl:1154) - Overwriting the .json content of 'Testsign-Release-Candidate:23' with new build model including promotion
      2019-07-24 21:36:48,188 [http-nio-12000-exec-19] [DEBUG] (o.a.b.BuildServiceImpl:661) - Starting to remove build 'Testsign-Release-Candidate' #23
      2019-07-24 21:38:50,684 [http-nio-12000-exec-19] [WARN ] (o.a.s.f.l.SessionLockEntry:132) - Mutable item 'artifactory-build-info:Testsign-Release-Candidate/23-1556834587668.json' has local modifications that will be discarded.
      2019-07-24 21:38:50,688 [http-nio-12000-exec-19] [WARN ] (o.a.s.f.l.SessionLockEntry:132) - Mutable item 'auto-trashcan:artifactory-build-info/Testsign-Release-Candidate/23-1556834587668.json' has local modifications that will be discarded.
      2019-07-24 21:38:50,744 [http-nio-12000-exec-19] [ERROR] (o.a.r.c.e.m.GlobalExceptionMapper:48) - Could not delete build 0 Testsign-Release-Candidate:23:2019-05-02T22:03:07.668Z
      org.artifactory.storage.StorageException: Could not delete build 0 Testsign-Release-Candidate:23:2019-05-02T22:03:07.668Z

       

      Also, noticed Database lock timeout error while promoting the build:

       

      com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Lock wait timeout exceeded; try restarting transaction

      After, troubleshooting in MySql DB with

      SHOW FULL  PROCESSLIST

      , noticed "Delete" operation was holding the lock and not releasing it, in turn leading to timeout. This issue has been reproduced in Postgres Db as well.

      Build promotion code requires to delete the build before the deploy:

      Delete is performed in a separate transaction:

      * We need to manually perform the delete BEFORE the deploy,
       * otherwise it can cause deploy-->delete instead of delete-->deploy.
       * This is because {@link SqlStorageSession} currently doesn't hold the order of events.
       *
       * @param build Build to be deleted
       */
       private void deletePreviousBuildAsSystem(Build build) {
       BasicStatusHolder status = new BasicStatusHolder();
       Authentication currentAuth = SecurityContextHolder.getContext().getAuthentication();
       try {
       SecurityContextHolder.getContext().setAuthentication(new SystemAuthenticationToken());
       BuildRun buildRun = getBuildRunInternally(build.getName(), build.getNumber(), build.getStarted());
       dbService.invokeInTransaction(
       "deleteBuildInPromotionBySystem" + build.getName() + build.getNumber() + build.getStarted(), () -> {
       getTransactionalMe().deleteBuild(buildRun, false, status);
       return null;
       });
       } finally {
       // Restore previous permissions
       SecurityContextHolder.getContext().setAuthentication(currentAuth);
       }
       if (status.hasErrors() || status.hasWarnings()) {
       log.warn("Couldn't delete build as part of promotion: " + status.getLastError().getMessage() + ".");
       }
       }

      There is no workaround for this issue. DB deadlock condition has been fixed in 6.11.5 version

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sowjanyak Sowjanya Kamatam
                Reporter:
                sowjanyak Sowjanya Kamatam
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: