Uploaded image for project: 'Artifactory Binary Repository'
  1. Artifactory Binary Repository
  2. RTFACT-16024

Running Debian metadata calculation causes too many threads to be used

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Normal
    • Resolution: Unresolved
    • Affects Version/s: 5.5.1, 5.9.1
    • Fix Version/s: None
    • Component/s: Debian
    • Labels:
      None

      Description

      Symptoms: Sometimes uploading a Debian file will give a 500 error, when uploading the file to a repository with many different existing Debian files.

      The error is an odd OutOfMemory Java exception. Artifactory does not crash, only the upload is refused.

      Steps to reproduce:
      1. Deploy 5k Debian files with different coordinates

      #!/bin/bash
      for i in

      {1..5000}

      do

      curl -uadmin:password -X PUT "http://localhost:8081/artifactory/deb-local/file$i.bz2;deb.distribution=trusty$i;deb.component=main;deb.architecture=i386"

      done

      2. Run POST to 'api/deb/reindex/deb-local'

      3. Monitor the java thread count and see how high it goes. In some systems, you might need to run the calculation multiple times (depending on number of thread limits configured in the OS)

      The error in the Artifactory logs:
      2018-03-02 16:19:38,088 [art-exec-31195] [ERROR] (o.a.w.q.WorkQueueImpl:85) - Debian Metadata: failed to process Debian Calculation Work Item '[deb_filename]'
      java.lang.RuntimeException: Failed to call work queue 'Debian Metadata' callback due to : unable to create new native thread
      at org.artifactory.work.queue.WorkQueueImpl.invoke(WorkQueueImpl.java:117) ~[artifactory-core-5.5.1-p001.jar:na]
      at org.artifactory.work.queue.WorkQueueImpl.doJobs(WorkQueueImpl.java:82) ~[artifactory-core-5.5.1-p001.jar:na]
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_151]
      at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_151]
      at org.artifactory.schedule.ArtifactoryConcurrentExecutor$RunnableWrapper.run(ArtifactoryConcurrentExecutor.java:104) ~[artifactory-storage-common-5.5.1-p001.jar:na]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_151]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_151]
      at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_151]
      Caused by: java.lang.reflect.InvocationTargetException: null
      at sun.reflect.GeneratedMethodAccessor511.invoke(Unknown Source) ~[na:na]
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_151]
      at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_151]
      at org.artifactory.work.queue.WorkQueueImpl.invoke(WorkQueueImpl.java:109) ~[artifactory-core-5.5.1-p001.jar:na]
      ... 7 common frames omitted
      Caused by: java.lang.OutOfMemoryError: unable to create new native thread

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            patrickr Patrick Russell
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Dates

              Created:
              Updated: