Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-19812

Access Token leak in HA cluster

    XMLWordPrintable

    Details

    • Severity:
      Critical

      Description

      Symptoms: An Access export times out when called by Artifactory. The logs indicate the "exporting tokens" is the cause of the delay, not users, groups, or permissions.

      This is a problem because long-running Artifactory HA clusters can have tens of thousands of tokens generated by the system. In one Artifactory cluster, the "access_tokens" table was observed to have 200,000 tokens, with 200 generated every day.

      Artifactory can fail to start if an Access Export times out.

      The workaround if Artifactory fails to start is to keep the .deleteForSecurityMarker file in the $ART_HOME/data or $CLUSTER_HOME/ha-data folder.

       

      Steps to reproduce:

      1.Deploy this debug logger to the $ART_HOME/etc/logback.xml:

          <logger name="org.artifactory.addon.ha">

             <level value="debug" />

          </logger>

       

      2. Create a repository on a secondary HA node, or make any other config descriptor change

       

      3. Observe that at least 2 new Access tokens are created in the database, and are never cleaned up:

      MySQL [artdb]> select count from access_tokens;
      ----------

      count

      ----------

      46

      ----------
      1 row in set (0.01 sec)

      [...]
      MySQL [artdb]> select count from access_tokens;
      ----------

      count

      ----------

      50

      ----------

       

      ------------------------------------------------------------------------------------++++----------------------------------------------------------------------------------------------------------------------------------------------------

      token_id                             subject                                         owner                           created       expiry        refresh_token scope                                                                  

      ------------------------------------------------------------------------------------++++----------------------------------------------------------------------------------------------------------------------------------------------------

      [...]

      RESTRICTED_TOKEN1 jfrt@01dhagrfexsvww1f5nn8xv1zyz/nodes/primary   jfrt@01dhagrfexsvww1f5nn8xv1zyz 1564795101416             0 NULL          NULL                                                                   
      RESTRICTED_TOKEN2 jfrt@01dhagrfexsvww1f5nn8xv1zyz/nodes/secondary jfrt@01dhagrfexsvww1f5nn8xv1zyz 1564795042467             0 NULL          NULL                                                                   

       

      Debug logs:

      2019-08-03 01:28:45,426 [http-nio-8081-exec-6] [INFO ] (o.a.u.r.s.a.c.r.CreateRepositoryConfigService:97) - Creating repository asdf5

      2019-08-03 01:28:45,433 [http-nio-8081-exec-6] [INFO ] (o.a.c.CentralConfigServiceImpl:615) - Reloading configuration... old revision 17, new revision 18

      2019-08-03 01:28:45,503 [http-nio-8081-exec-6] [INFO ] (o.a.c.CentralConfigServiceImpl:374) - New configuration with revision 18 saved.

      2019-08-03 01:28:45,505 [http-nio-8081-exec-6] [INFO ] (o.a.s.ArtifactoryApplicationContext:515) - Artifactory application context set to NOT READY by reload

      2019-08-03 01:28:45,578 [http-nio-8081-exec-6] [INFO ] (o.a.s.BaseTaskServiceDescriptorHandler:51) - No Replication configured. Replication is disabled.

      2019-08-03 01:28:45,608 [http-nio-8081-exec-6] [DEBUG] (o.a.a.h.p.HaPropagationServiceImpl:553) - Created new token: Bearer

      2019-08-03 01:28:45,625 [http-nio-8081-exec-6] [INFO ] (o.a.s.ArtifactoryApplicationContext:515) - Artifactory application context set to READY by reload

      2019-08-03 01:28:45,626 [http-nio-8081-exec-6] [INFO ] (o.a.c.CentralConfigServiceImpl:633) - Configuration reloaded.

      2019-08-03 01:28:45,629 [http-nio-8081-exec-6] [DEBUG] (o.a.a.h.p.HaPropagationServiceImpl:270) - Propagation started for [primary]

      2019-08-03 01:28:45,630 [art-exec-171] [DEBUG] (o.a.a.h.p.HaPropagationServiceImpl:328) - Propagating 'central config change' to 'primary (http://172.20.9.4:8081/artifactory/api/ha/descriptor/change/)'

      2019-08-03 01:28:45,630 [art-exec-171] [DEBUG] (o.a.a.h.p.HaPropagationServiceImpl:438) - Added token to header: X-Artifactory-HA-Security-Token

      2019-08-03 01:28:45,676 [art-exec-171] [DEBUG] (o.a.a.h.p.HaPropagationServiceImpl:345) - Successfully propagate event 'central config change' to 'primary'

        Attachments

          Activity

            People

            Assignee:
            elis Eli Skoran
            Reporter:
            patrickr Patrick Russell
            Votes:
            1 Vote for this issue
            Watchers:
            10 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved:

                Sync Status

                Connection: RTFACT Sync
                RTMID-19812 -
                SYNCHRONIZED
                • Last Sync Date: