Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-16517

Gem artifacts with dependency queries too long can’t be resolved from virtual repositories

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Normal
    • Resolution: Fixed
    • Affects Version/s: 5.10.3, 5.8.9
    • Fix Version/s: 6.12.0, 6.11.3
    • Component/s: None
    • Labels:

      Description

      Certain Artifacts can't be resolved from a gems virtual repository due to their dependencies creating too long of a query parameter list. 

      This affect Gem versions > 2.1

      One example is package aws-sdk. Note how the GET request has an incredibly long query parameter list.

      Reproduce steps:

      $ gem install aws-sdk --verbose
      
      Getting SRV record failed: DNS result has no information for _rubygems._tcp.localhost
      
      HEAD http://localhost:8081/artifactory/api/gems/gems/api/v1/dependencies
      
      200 OK
      
      GET http://localhost:8081/artifactory/api/gems/gems/api/v1/dependencies?gems=aws-sdk
      
      200 OK
      
      Getting SRV record failed: DNS result has no information for _rubygems._tcp.localhost
      
      GET http://localhost:8081/artifactory/api/gems/gems/api/v1/dependencies?gems=aws-sdk-resources
      
      200 OK
      
      GET http://localhost:8081/artifactory/api/gems/gems/api/v1/dependencies?gems=aws-sdk-acm,aws-sdk-acmpca,aws-sdk-alexaforbusiness,aws-sdk-apigateway,aws-sdk-applicationautoscaling,aws-sdk-applicationdiscoveryservice,aws-sdk-appstream,aws-sdk-appsync,aws-sdk-athena,aws-sdk-autoscaling,aws-sdk-autoscalingplans,aws-sdk-batch,aws-sdk-budgets,aws-sdk-cloud9,aws-sdk-clouddirectory,aws-sdk-cloudformation,aws-sdk-cloudfront,aws-sdk-cloudhsm,aws-sdk-cloudhsmv2,aws-sdk-cloudsearch,aws-sdk-cloudsearchdomain,aws-sdk-cloudtrail,aws-sdk-cloudwatch,aws-sdk-cloudwatchevents,aws-sdk-cloudwatchlogs,aws-sdk-codebuild,aws-sdk-codecommit,aws-sdk-codedeploy,aws-sdk-codepipeline,aws-sdk-codestar,aws-sdk-cognitoidentity,aws-sdk-cognitoidentityprovider,aws-sdk-cognitosync,aws-sdk-comprehend,aws-sdk-configservice,aws-sdk-connect,aws-sdk-costandusagereportservice,aws-sdk-costexplorer,aws-sdk-databasemigrationservice,aws-sdk-datapipeline,aws-sdk-dax,aws-sdk-devicefarm,aws-sdk-directconnect,aws-sdk-directoryservice,aws-sdk-dynamodb,aws-sdk-dynamodbstreams,aws-sdk-ec2,aws-sdk-ecr,aws-sdk-ecs,aws-sdk-efs,aws-sdk-elasticache,aws-sdk-elasticbeanstalk,aws-sdk-elasticloadbalancing,aws-sdk-elasticloadbalancingv2,aws-sdk-elasticsearchservice,aws-sdk-elastictranscoder,aws-sdk-emr,aws-sdk-firehose,aws-sdk-fms,aws-sdk-gamelift,aws-sdk-glacier,aws-sdk-glue,aws-sdk-greengrass,aws-sdk-guardduty,aws-sdk-health,aws-sdk-iam,aws-sdk-importexport,aws-sdk-inspector,aws-sdk-iot,aws-sdk-iotanalytics,aws-sdk-iotdataplane,aws-sdk-iotjobsdataplane,aws-sdk-kinesis,aws-sdk-kinesisanalytics,aws-sdk-kinesisvideo,aws-sdk-kinesisvideoarchivedmedia,aws-sdk-kinesisvideomedia,aws-sdk-kms,aws-sdk-lambda,aws-sdk-lambdapreview,aws-sdk-lex,aws-sdk-lexmodelbuildingservice,aws-sdk-lightsail,aws-sdk-machinelearning,aws-sdk-marketplacecommerceanalytics,aws-sdk-marketplaceentitlementservice,aws-sdk-marketplacemetering,aws-sdk-mediaconvert,aws-sdk-medialive,aws-sdk-mediapackage,aws-sdk-mediastore,aws-sdk-mediastoredata,aws-sdk-migrationhub,aws-sdk-mobile,aws-sdk-mq,aws-sdk-mturk,aws-sdk-opsworks,aws-sdk-opsworkscm,aws-sdk-organizations,aws-sdk-pinpoint,aws-sdk-polly,aws-sdk-pricing,aws-sdk-rds,aws-sdk-redshift,aws-sdk-rekognition,aws-sdk-resourcegroups,aws-sdk-resourcegroupstaggingapi,aws-sdk-route53,aws-sdk-route53domains,aws-sdk-s3,aws-sdk-sagemaker,aws-sdk-sagemakerruntime,aws-sdk-secretsmanager,aws-sdk-serverlessapplicationrepository,aws-sdk-servicecatalog,aws-sdk-servicediscovery,aws-sdk-ses,aws-sdk-shield,aws-sdk-simpledb,aws-sdk-sms,aws-sdk-snowball,aws-sdk-sns,aws-sdk-sqs,aws-sdk-ssm,aws-sdk-states,aws-sdk-storagegateway,aws-sdk-support,aws-sdk-swf,aws-sdk-transcribeservice,aws-sdk-translate,aws-sdk-waf,aws-sdk-wafregional,aws-sdk-workdocs,aws-sdk-workmail,aws-sdk-workspaces,aws-sdk-xray
      
      200 OK
      
      ERROR:  Could not find a valid gem 'aws-sdk-acm' (~> 1) in any repository
      
      GET http://localhost:8081/artifactory/api/gems/gems/latest_specs.4.8.gz
      
      304 Not Modified
      
      ERROR:  Possible alternatives: aws-sdk-acm
      
      
      

      Works with remote, and seems the 502 is not handled well on virtual:

      $ gem install aws-sdk --verbose
      
      Getting SRV record failed: DNS result has no information for _rubygems._tcp.localhost
      
      HEAD http://localhost:8081/artifactory/api/gems/gems-remote/api/v1/dependencies
      
      200 OK
      
      GET http://localhost:8081/artifactory/api/gems/gems-remote/api/v1/dependencies?gems=aws-sdk
      
      200 OK
      
      Getting SRV record failed: DNS result has no information for _rubygems._tcp.localhost
      
      GET http://localhost:8081/artifactory/api/gems/gems-remote/quick/Marshal.4.8/aws-sdk-3.0.1.gemspec.rz
      
      200 OK
      
      GET http://localhost:8081/artifactory/api/gems/gems-remote/api/v1/dependencies?gems=aws-sdk-resources
      
      200 OK
      
      GET http://localhost:8081/artifactory/api/gems/gems-remote/api/v1/dependencies?gems=aws-sdk-acm,aws-sdk-acmpca,aws-sdk-alexaforbusiness,aws-sdk-apigateway,aws-sdk-applicationautoscaling,aws-sdk-applicationdiscoveryservice,aws-sdk-appstream,aws-sdk-appsync,aws-sdk-athena,aws-sdk-autoscaling,aws-sdk-autoscalingplans,aws-sdk-batch,aws-sdk-budgets,aws-sdk-cloud9,aws-sdk-clouddirectory,aws-sdk-cloudformation,aws-sdk-cloudfront,aws-sdk-cloudhsm,aws-sdk-cloudhsmv2,aws-sdk-cloudsearch,aws-sdk-cloudsearchdomain,aws-sdk-cloudtrail,aws-sdk-cloudwatch,aws-sdk-cloudwatchevents,aws-sdk-cloudwatchlogs,aws-sdk-codebuild,aws-sdk-codecommit,aws-sdk-codedeploy,aws-sdk-codepipeline,aws-sdk-codestar,aws-sdk-cognitoidentity,aws-sdk-cognitoidentityprovider,aws-sdk-cognitosync,aws-sdk-comprehend,aws-sdk-configservice,aws-sdk-connect,aws-sdk-costandusagereportservice,aws-sdk-costexplorer,aws-sdk-databasemigrationservice,aws-sdk-datapipeline,aws-sdk-dax,aws-sdk-devicefarm,aws-sdk-directconnect,aws-sdk-directoryservice,aws-sdk-dynamodb,aws-sdk-dynamodbstreams,aws-sdk-ec2,aws-sdk-ecr,aws-sdk-ecs,aws-sdk-efs,aws-sdk-elasticache,aws-sdk-elasticbeanstalk,aws-sdk-elasticloadbalancing,aws-sdk-elasticloadbalancingv2,aws-sdk-elasticsearchservice,aws-sdk-elastictranscoder,aws-sdk-emr,aws-sdk-firehose,aws-sdk-fms,aws-sdk-gamelift,aws-sdk-glacier,aws-sdk-glue,aws-sdk-greengrass,aws-sdk-guardduty,aws-sdk-health,aws-sdk-iam,aws-sdk-importexport,aws-sdk-inspector,aws-sdk-iot,aws-sdk-iotanalytics,aws-sdk-iotdataplane,aws-sdk-iotjobsdataplane,aws-sdk-kinesis,aws-sdk-kinesisanalytics,aws-sdk-kinesisvideo,aws-sdk-kinesisvideoarchivedmedia,aws-sdk-kinesisvideomedia,aws-sdk-kms,aws-sdk-lambda,aws-sdk-lambdapreview,aws-sdk-lex,aws-sdk-lexmodelbuildingservice,aws-sdk-lightsail,aws-sdk-machinelearning,aws-sdk-marketplacecommerceanalytics,aws-sdk-marketplaceentitlementservice,aws-sdk-marketplacemetering,aws-sdk-mediaconvert,aws-sdk-medialive,aws-sdk-mediapackage,aws-sdk-mediastore,aws-sdk-mediastoredata,aws-sdk-migrationhub,aws-sdk-mobile,aws-sdk-mq,aws-sdk-mturk,aws-sdk-opsworks,aws-sdk-opsworkscm,aws-sdk-organizations,aws-sdk-pinpoint,aws-sdk-polly,aws-sdk-pricing,aws-sdk-rds,aws-sdk-redshift,aws-sdk-rekognition,aws-sdk-resourcegroups,aws-sdk-resourcegroupstaggingapi,aws-sdk-route53,aws-sdk-route53domains,aws-sdk-s3,aws-sdk-sagemaker,aws-sdk-sagemakerruntime,aws-sdk-secretsmanager,aws-sdk-serverlessapplicationrepository,aws-sdk-servicecatalog,aws-sdk-servicediscovery,aws-sdk-ses,aws-sdk-shield,aws-sdk-simpledb,aws-sdk-sms,aws-sdk-snowball,aws-sdk-sns,aws-sdk-sqs,aws-sdk-ssm,aws-sdk-states,aws-sdk-storagegateway,aws-sdk-support,aws-sdk-swf,aws-sdk-transcribeservice,aws-sdk-translate,aws-sdk-waf,aws-sdk-wafregional,aws-sdk-workdocs,aws-sdk-workmail,aws-sdk-workspaces,aws-sdk-xray
      
      502 Bad Gateway
      
      GET http://localhost:8081/artifactory/api/gems/gems-remote/prerelease_specs.4.8.gz
      
      200 OK
      
      GET http://localhost:8081/artifactory/api/gems/gems-remote/specs.4.8.gz
      
      304 Not Modified
      
      GET http://localhost:8081/artifactory/api/gems/gems-remote/quick/Marshal.4.8/aws-sdk-acm-1.0.0.gemspec.rz
      
      200 OK
      
      
      

      Debug loggers don't give extra information, it could be something related to that 502 error, as that's the big difference in the outputs. 

       

      The long URL through cURL or Browser gives out the following for remote repository requests:

          <h1>Error 502 Bad Gateway</h1>

          <p>Bad Gateway</p>

          <h3>Guru Mediation:</h3>

          <p>Details: cache-sjc3633-SJC 1525229284 1719933408</p>

          <hr>

          <p>Varnish cache server</p>

      For Virtual repositories, it simply returns a close bracket symbol " ] ".

       

        Attachments

          Activity

            People

            • Assignee:
              omriz Omri Ziv
              Reporter:
              angellom Angello Maggio
              Assigned QA:
              Andrey Tashlitsky
            • Votes:
              7 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: