Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-26220

Unable to resolve GitHub NPM remote packages when pointing NPM client to virtual repository

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: In Progress
    • Priority: 2 - Critical
    • Resolution: Unresolved
    • Affects Version/s: 7.16.3
    • Fix Version/s: None
    • Component/s: Artifactory, NPM
    • Labels:
    • Environment:
    • Location:
      External

      Description

      Problem description:

      When the NPM client is pointing to a virtual NPM repository and if the virtual repository contains a remote npm repository that is pointing to “https://npm.pkg.github.com/” then we will see a 404 response while resolving the GitHub packages.

      The client output is as follows:

      npm verb stack HttpErrorGeneral: 404 Not Found - GET http://10.168.0.111:12362/artifactory/api/npm/santhosh-npm/download/@santhoshjfrog/ajv/6.12.6/69932300ed5c7037083583eb57560ba68a7decb7ee601654d051c95bbc01ca2b
      npm verb stack     at /usr/local/lib/node_modules/npm/node_modules/npm-registry-fetch/check-response.js:95:15
      npm verb stack     at processTicksAndRejections (node:internal/process/task_queues:96:5)
      npm verb statusCode 404
      npm verb pkgid @santhoshjfrog/ajv@http://10.168.0.111:12362/artifactory/api/npm/santhosh-npm/download/@santhoshjfrog/ajv/6.12.6/69932300ed5c7037083583eb57560ba68a7decb7ee601654d051c95bbc01ca2b
      npm verb cwd /npm
      npm verb Linux 5.4.0-1049-gcp
      npm verb argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "@santhoshjfrog/ajv@6.12.6" "--verbose"
      npm verb node v16.6.1
      npm verb npm  v7.20.3
      npm ERR! code E404
      npm ERR! 404 Not Found - GET http://10.168.0.111:12362/artifactory/api/npm/santhosh-npm/download/@santhoshjfrog/ajv/6.12.6/69932300ed5c7037083583eb57560ba68a7decb7ee601654d051c95bbc01ca2b
      npm ERR! 404 
      npm ERR! 404  '@santhoshjfrog/ajv@http://10.168.0.111:12362/artifactory/api/npm/santhosh-npm/download/@santhoshjfrog/ajv/6.12.6/69932300ed5c7037083583eb57560ba68a7decb7ee601654d051c95bbc01ca2b' is not in the npm registry.
      npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
      npm ERR! 404 
      npm ERR! 404 Note that you can also install from a
      npm ERR! 404 tarball, folder, http url, or git url.
      npm verb exit 1
      npm timing npm Completed in 523ms
      npm verb unfinished npm timer reify 1628212923422
      npm verb unfinished npm timer reify:audit 1628212923594
      npm verb unfinished npm timer auditReport:getReport 1628212923595
      npm verb unfinished npm timer reify:unpack 1628212923598
      npm verb unfinished npm timer reifyNode:node_modules/@santhoshjfrog/ajv 1628212923598
      npm verb code 1
       
      npm ERR! A complete log of this run can be found in:
      npm ERR!     /root/.npm/_logs/2021-08-06T01_22_03_714Z-debug.log
      

      If we point the NPM client to the remote NPM repository, then we are able to download the packages from GitHub successfully.

      npm http fetch GET 200 http://10.168.0.111:12162/artifactory/api/npm/santhosh-npm/@santhoshjfrog%2fajv 1513ms (cache miss)
      npm http fetch GET 200 https://registry.npmjs.org/fast-deep-equal 15ms (cache hit)
      npm http fetch GET 200 https://registry.npmjs.org/json-schema-traverse 14ms (cache hit)
      npm http fetch GET 200 https://registry.npmjs.org/uri-js 15ms (cache hit)
      npm http fetch GET 200 https://registry.npmjs.org/fast-json-stable-stringify 17ms (cache hit)
      npm timing idealTree:#root Completed in 1542ms
      npm http fetch GET 200 https://registry.npmjs.org/punycode 5ms (cache hit)
      npm timing idealTree:node_modules/@santhoshjfrog/ajv Completed in 20ms
      npm timing idealTree:node_modules/fast-deep-equal Completed in 0ms
      npm timing idealTree:node_modules/fast-json-stable-stringify Completed in 0ms
      npm timing idealTree:node_modules/json-schema-traverse Completed in 0ms
      npm timing idealTree:node_modules/uri-js Completed in 3ms
      npm timing idealTree:node_modules/punycode Completed in 0ms
      npm timing idealTree:buildDeps Completed in 1568ms
      npm timing idealTree:fixDepFlags Completed in 0ms
      npm timing idealTree Completed in 1589ms
      npm timing reify:loadTrees Completed in 1590ms
      npm timing reify:diffTrees Completed in 1ms
      npm timing reify:retireShallow Completed in 0ms
      npm timing reify:createSparse Completed in 4ms
      npm timing reify:loadBundles Completed in 0ms
      npm timing reifyNode:node_modules/punycode Completed in 61ms
      npm timing reifyNode:node_modules/json-schema-traverse Completed in 67ms
      npm timing reifyNode:node_modules/fast-deep-equal Completed in 67ms
      npm timing reifyNode:node_modules/fast-json-stable-stringify Completed in 69ms
      npm timing reifyNode:node_modules/uri-js Completed in 82ms
      npm http fetch POST 200 https://registry.npmjs.org/-/npm/v1/security/advisories/bulk 177ms
      npm timing auditReport:getReport Completed in 179ms
      npm timing auditReport:init Completed in 0ms
      npm timing reify:audit Completed in 180ms
      npm http fetch GET 200 http://10.168.0.111:12162/artifactory/api/npm/santhosh-npm/download/@santhoshjfrog/ajv/6.12.6/69932300ed5c7037083583eb57560ba68a7decb7ee601654d051c95bbc01ca2b 1216ms (cache miss)
      npm timing reifyNode:node_modules/@santhoshjfrog/ajv Completed in 1238ms
      npm timing reify:unpack Completed in 1239ms
      npm timing reify:unretire Completed in 0ms
      npm timing build:queue Completed in 4ms
      npm timing build:deps Completed in 4ms
      npm timing build Completed in 4ms
      npm timing reify:build Completed in 6ms
      npm timing reify:trash Completed in 0ms
      npm timing reify:save Completed in 6ms
      npm timing reify Completed in 2883ms
      added 6 packages, and audited 7 packages in 3s
      found 0 vulnerabilities
      npm timing command:install Completed in 2889ms
      npm verb exit 0
      npm timing npm Completed in 3110ms
      npm info ok 
      

      The issue started happening from Artifactory version 7.16.3:https://www.jfrog.com/confluence/display/JFROG/Artifactory+Release+Notes#ArtifactoryReleaseNotes-Artifactory7.16.3. We are able to download the package successfully while pointing to the virtual repository before version 7.16.3.

      I tested in the below version and the results are as follows:

      Artifactory 7.12.5 → Working

      Artifactory 7.15.5 → Working

      Artifactory 7.16.3 → Not working

      Artifactory 7.23.3 → Not working

      workaround solution, for now, is to add the .npmrc as follows:

       

      registry=https://test.jfrog.io/artifactory/api/npm/github-npm-remote/
      always-auth=true
      @<scope>:registry=https://test.jfrog.io/artifactory/api/npm/github-npm-remote/
      //test.jfrog.io/artifactory/api/npm/github-npm-remote/:_password=<password>;
      //test.jfrog.io/artifactory/api/npm/github-npm-remote/:username=a@a.com
      email=a@a.com
      registry=https://test.jfrog.io/artifactory/api/npm/default-npm-virtual/
      always-auth=true
      //test.jfrog.io/artifactory/api/npm/default-npm-virtual/:_password=<password>;
      //test.jfrog.io/artifactory/api/npm/default-npm-virtual/:username=a@a.com
      email=a@a.com
      

      This makes users download the GitHub scoped package from the remote repo and the dependencies from the NPMjs.

      What is the expected behavior? 

      We should be able to resolve the package that is available in the GitHub NPM remote repository when the NPM client points to the NPM virtual repository which contains the GitHub NPM remote repository.

      Steps to reproduce:

      1. Set up an Artifactory instance (Version 7.16.3 and above)
      2. Create an npm-local, npm-remote (pointing to https://registry.npmjs.org),npm-github-remote (pointing to https://npm.pkg.github.com/ ) and npm virtual repository.
      3. All npm-local, npm-remote, and  npm-github-remote to npm virtual repository
      4. Make sure you have private packages in your Github (https://npm.pkg.github.com/) : https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-npm-registry#publishing-a-package
      5. In Artifactory, add GitHub username and personal access token in the Remote authentication section of your npm-github-remote (pointing to https://npm.pkg.github.com/ ) remote repository.
      6. Enable “ Bypass HEAD Requests” for the npm-github-remote remote repository.
      7. Now from the terminal, using NPM client run the below commands (pointing to NPM virtual):
      8. npm config set @<github-owner>:registry[ http://10.168.0.111:8081/artifactory/api/npm/npm/|http://mill.jfrog.team:12511/artifactory/api/npm/santhosh-npm/]
      9. npm login --scope=@<github-owner> --registry=http://10.168.0.111:8081/artifactory/api/npm/npm/
      10. npm install @<github-owner>/<package-name> --verbose

       

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            santhoshp Santhosh Pesari
            Votes:
            2 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:

                Sync Status

                Connection: RTFACT Sync
                RTMID-26221 -
                SYNCHRONIZED
                • Last Sync Date: