Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-22284

JFrog Artifactory intermittently rejects authentication with 403 forbidden

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: 3 - High
    • Resolution: Unresolved
    • Affects Version/s: 7.4.3
    • Fix Version/s: None
    • Component/s: RPM
    • Environment:

      JFrog Platform 7.4.3 running in Kubernetes

    • Severity:
      Medium

      Description

      Good day,

      We would like your assistance to check on a possible bug with JFrog Artifactory 7.4.

      We are having a hard time configuring zypper clients (SLES and OpenSUSE) to work with our RPM Repository due to some "authentication" issues. At first, we thought it was just an issue with zypper and opensuse so we first focused on our efforts to make things work with zypper.

      Error:

      opensuse:~ # zypper refresh myapp
      Retrieving repository 'myapp' metadata ....................................................................................................................................[error]
      Repository 'myapp' is invalid.
      [myapp|https://myuser@rpm.example.com/myapp-release-rpm/stable/myapp/2019/3/] Valid metadata not found at specified URL
      History:
       - [myapp|https://myuser@rpm.example.com/myapp-release-rpm/stable/myapp/2019/3/] Repository type can't be determined.
      
      Please check if the URIs defined for this repository are pointing to a valid repository.
      Skipping repository 'myapp' because of the above error.
      Could not refresh the repositories because of errors.
      opensuse:~ #
      

      But these authentication-related errors started to come up every now and then with yum clients as well (centOS), although with YUM it is very rare.

      In a StackOverflow post, I described the issue in more detail.


      So I tried doing man-in-the-middle to see what's happening under the hood, and this is the sequence with zypper:

      Scenario 1 - successful authentication

      The following is the sequence of a zypper refresh --repo myrepo:

      (1) zypper sends an HTTP HEAD request with the base64-encoded username::

      HEAD /myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml HTTP/1.1
      Host: rpm.example.com
      Authorization: Basic dXNlcm5hbWU6
      User-Agent: ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      Accept: */*
      Connection: close
      

      (2) jfrog responds with HTTP 401 Unauthorized with the WWW-Authenticate header:

      HTTP/1.1 401 Unauthorized
      Date: Thu, 28 May 2020 08:20:04 GMT
      Content-Type: application/json;charset=ISO-8859-1
      Connection: close
      Server: Artifactory/7.4.3 70403900
      X-Artifactory-Id: 2148103ba10eacbb:-16f1c4c1:172093a231a:-8000
      X-Artifactory-Node-Id: artifactory-server
      WWW-Authenticate: Basic realm="Artifactory Realm"
      

      (3) zypper sends another HTTP HEAD request, this time with the base64-encoded username:password:

      HEAD /myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml HTTP/1.1
      Host: rpm.example.com
      Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
      User-Agent: ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      Accept: */*
      Connection: close
      

      (4) jfrog finally responds with an HTTP 200.

      HTTP/1.1 200 OK
      Date: Thu, 28 May 2020 08:20:04 GMT
      Content-Type: application/xml
      Content-Length: 1394
      Connection: close
      Server: Artifactory/7.4.3 70403900
      X-Artifactory-Id: 2148103ba10eacbb:-16f1c4c1:172093a231a:-8000
      X-Artifactory-Node-Id: artifactory-server
      Last-Modified: Fri, 08 May 2020 10:25:19 GMT
      Accept-Ranges: bytes
      X-Artifactory-Filename: repomd.xml
      Cache-Control: no-store
      

      These are logged by Artifactory:

      artifactory-request.log:

      2020-05-28T08:20:34.566Z [5f78297c2aeabaa8] [DENIED LOGIN]   for client : username / 213.1.1.1. 
      2020-05-28T08:20:34.870Z [570978212a5318e3] [ACCEPTED DOWNLOAD] myapp-release-rpm-cache:stable/myapp/2019/3/repodata/repomd.xml  for client : username / 213.1.1.1.
      

      artifactory-access.log:

      2020-05-28T08:20:34.566Z|5f78297c2aeabaa8|213.2.2.2|non_authenticated_user|HEAD|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|401|-1|0|8|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      2020-05-28T08:20:34.721Z|8018b7cbc9c424e8|213.2.2.2|username|HEAD|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|200|-1|1394|3|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      2020-05-28T08:20:34.870Z|570978212a5318e3|213.2.2.2|username|GET|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|200|-1|1394|2|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      ...
      

      So basically, zypper comes in with a HEAD request, JFrog says "you're not authenticated", zypper responds back and tries to authenticate, and finally JFrog authenticates zypper. Makes sense so far.

      Scenario 2 - 403 forbidden

      Do some work, then run the same zypper refresh --repo myrepo command after a few minutes or so, and here is the result:

      (1) zypper sends an HTTP HEAD request with the base64-encoded username::

      HEAD /myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml HTTP/1.1
      Host: rpm.example.com
      Authorization: Basic dXNlcm5hbWU6
      User-Agent: ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      Accept: */*
      Connection: close
      

      (2) jfrog responds with HTTP 401 Unauthorized with the WWW-Authenticate header:

      HTTP/1.1 401 Unauthorized
      Date: Thu, 28 May 2020 08:30:44 GMT
      Content-Type: application/json;charset=ISO-8859-1
      Connection: close
      Server: Artifactory/7.4.3 70403900
      X-Artifactory-Id: 2148103ba10eacbb:-16f1c4c1:172093a231a:-8000
      X-Artifactory-Node-Id: artifactory-server
      WWW-Authenticate: Basic realm="Artifactory Realm"
      

      (3) zypper sends another HTTP HEAD request, this time with the base64-encoded username:password:

      HEAD /myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml HTTP/1.1
      Host: rpm.example.com
      Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=
      User-Agent: ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      Accept: */*
      Connection: close
      

      (4) this time, jfrog responds with 403 Forbidden instead of 200 OK.

      HTTP/1.1 403 Forbidden
      Date: Thu, 28 May 2020 08:30:44 GMT
      Content-Type: application/json;charset=ISO-8859-1
      Connection: close
      Server: Artifactory/7.4.3 70403900
      X-Artifactory-Id: 2148103ba10eacbb:-16f1c4c1:172093a231a:-8000
      X-Artifactory-Node-Id: artifactory-server
      WWW-Authenticate: Basic realm="Artifactory Realm"
      

      artifactory-request.log:

      2020-05-28T08:30:44.496Z [46c81a2450623166] [DENIED LOGIN]   for client : username / 213.1.1.1.
      2020-05-28T08:30:44.630Z [769ed41c652daa7a] [DENIED LOGIN]   for client : username / 213.1.1.1.
      

      artifactory-access.log:

      2020-05-28T08:30:44.496Z|46c81a2450623166|213.2.2.2|non_authenticated_user|HEAD|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|401|-1|0|9|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      2020-05-28T08:30:44.630Z|769ed41c652daa7a|213.2.2.2|non_authenticated_user|HEAD|/myapp-release-rpm/stable/myapp/2019/3/repodata/repomd.xml|403|-1|0|1|ZYpp 17.19.0 (curl 7.60.0) openSUSE-Leap-15.1-x86_64
      

      Notice that zypper sends the same Authorization header value when asked to authenticate, but on the second scenario, JFrog fails to authenticate the request.

      Did anybody have this same issue with JFrog before? We are guessing this is an issue with JFrog 7 since ours was just recently upgraded, but there is no way for us to verify this. And unfortunately for us, we are on the paid plan which doesn't even have a support license.

      Any suggestions and comments will be very much appreciated.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              Unassigned
              Reporter:
              lg8977 Lester Guerzon
              Votes:
              13 Vote for this issue
              Watchers:
              18 Start watching this issue

                Dates

                Created:
                Updated:

                  Sync Status

                  Connection: RTFACT Sync
                  RTMID-22284 -
                  SYNCHRONIZED
                  • Last Sync Date: