Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-18626

NPM dependency rewrites does not account for "github:user/package" format

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.0.0, 6.17.0
    • Component/s: None
    • Labels:

      Description

      Currently, we support the following resolvers for npm dependencies on git:

      git+ssh://user@hostname:project.git#commit-ish
      git+ssh://user@hostname/project.git#commit-ish
      git+https://git@github.com/<user>/<filename>.git
      user/repository
      

      However, some packages have the format "github:user/repository", such as jspdf, and its dependency file-saver:

      "dependencies": {
          "canvg": "1.5.3",
          "file-saver": "github:eligrey/FileSaver.js#1.3.8",
          "html2canvas": "1.0.0-alpha.12",
          "omggif": "1.0.7",
          "promise-polyfill": "8.1.0",
          "stackblur-canvas": "2.2.0"
        },
      

      As you can see, there is a "github:" right before file-saver, which seems to confuse Artifactory, as when I enabled debug loggers:

      2019-02-27 02:10:42,547 [http-nio-8081-exec-51] [DEBUG] (o.a.a.n.r.r.e.NpmDependencyFetcherFactory:51) - 'github:eligrey/FileSaver.js#1.3.8' is not a valid URL.
      

      It does not try to resolve via Artifactory at all, and rewrite the dependency. It only goes through to github. I verified this by blocking resolution to github.com and received an error on the ls remote:

      npm ERR! Error while executing:
      npm ERR! /usr/bin/git ls-remote -h -t ssh://git@github.com/eligrey/FileSaver.js.git
      npm ERR! 
      npm ERR! Host key verification failed.
      npm ERR! fatal: Could not read from remote repository.
      npm ERR! 
      npm ERR! Please make sure you have the correct access rights
      npm ERR! and the repository exists.
      npm ERR! 
      npm ERR! exited with error code: 128
      

      and it will not cache in _external either. I tried creating a test package with the same dependency sans prefix:

      $ cat package.json 
      {
        "name": "file-saver-test",
        "version": "1.0.1",
        "description": "this is a package that pulls file-saver",
        "main": "index.js",
        "scripts": {
          "test": "echo \"Error: no test specified\" && exit 1"
        },
        "dependencies": {
          "file-saver": "eligrey/FileSaver.js#1.3.8"
        },
        "author": "",
        "license": "ISC"
      }
      

      And tried resolving that from Artifactory, and it proved successful (file-saver's package.json snippet):

      {
        "_from": "file-saver@eligrey-FileSaver.js-1.3.8.tar.gz-art-external",
        "_id": "file-saver@1.3.8",
        "_inBundle": false,
        "_integrity": "sha1-PJ+5uWOsJ/k3nsf7p02TfZJEeps=",
        "_location": "/file-saver",
        "_phantomChildren": {},
        "_requested": {
          "type": "tag",
          "registry": true,
          "raw": "file-saver@eligrey-FileSaver.js-1.3.8.tar.gz-art-external",
          "name": "file-saver",
          "escapedName": "file-saver",
          "rawSpec": "eligrey-FileSaver.js-1.3.8.tar.gz-art-external",
          "saveSpec": null,
          "fetchSpec": "eligrey-FileSaver.js-1.3.8.tar.gz-art-external"
        },
        "_requiredBy": [
          "/file-saver-test"
        ],
        "_resolved": "http://localhost:8081/artifactory/api/npm/npm/_external/eligrey-FileSaver.js-1.3.8.tar.gz",
        "_shasum": "3c9fb9b963ac27f9379ec7fba74d937d92447a9b",
        "_spec": "file-saver@eligrey-FileSaver.js-1.3.8.tar.gz-art-external",
      

      and we see it listed under the _external directory of our npm-remote-cache:

      Index of npm-remote-cache/_external
      Name                                                          Last modified      Size
      ../
      eligrey-FileSaver.js-1.3.8.tar.gz                              27-Feb-2019 08:03  26.74 KB
      json-schema-JSON-Schema-Test-Suite-master.tar.gz               27-Feb-2019 08:07  45.30 KB
      ronn-master                                                    28-Feb-2019 01:00  19.78 KB
      rynomad-jsbn-master.tar.gz                                     27-Feb-2019 08:07  11.53 KB
      tc39-test262-18c1e799a01cc976695983b61e225ce7959bdd91.tar.gz   27-Feb-2019 08:06  3.99 MB
      tc39-test262-3bfad28cc302fd4455badcfcbca7c5bb7ce41a72.tar.gz   27-Feb-2019 08:06  4.09 MB
      tc39-test262-51553973738063f457e248f7f1e643c561c8a64c.tar.gz   27-Feb-2019 08:06  3.95 MB
      TreehouseJS-CSSStyleDeclaration-master.tar.gz                  27-Feb-2019 08:06  27.11 KB
      

      Unfortunately, we haven't been able to find a workaround yet, due to the fact that it is a dependency of a dependency, as it would require modification of the package.json, as described above, and remove the 'github' specifier for each dependency in order for it to work. We dove into the NPM documentation, https://docs.npmjs.com/files/package.json#github-urls, and it does not explicitly say that "gitbub:user/package" is a legitimate syntax for dependency co-ordinates, but it is referenced online in passing, such as in this SO link:
      https://stackoverflow.com/questions/23210437/npm-install-private-github-repositories-by-dependency-in-package-json

      as well as the fact that the npm client is able to handle it. This was tested on a number of Artifactory versions, including 6.3.2, 6.7.3 and 6.8.3 (latest). To reproduce:
      enable NPM dependency rewrites
      block github.com access (optional)
      npm install jspdf@1.5.3

      see that file-saver is not cached in the _external repository

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                rotemk Rotem Kfir
                Reporter:
                loreny Loren Yeung
                Assigned QA:
                Bar Haim
              • Votes:
                3 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: