Eventual Queue waiting for the first file to be uploaded to S3 before deploying other files.
This means all other files deployed will wait in line before being available on the s3, and will also remain in the _queue directory until the first file is uploaded.
- Prepare 3 files, the 1st file “30g.dat”(size=30GB); 2nd file “1k.dat”(size=1KB); 3rd file “2k.dat”(size=2KB)
- Upload the file “30g.dat” to Artifactory successfully.
- Wait for 15s, make sure the dispatcher already dispatched it to some worker.
- Upload file “1k.dat” to Artifactory successfully.
- Wait for 15s, check the eventual _queue folder: <artifactory_data_dir>/eventual/_queue/, we expect the “1k.dat” should be scheduled to some idle worker after 1s and transfer it to s3 server, then remove it from _queue folder.
- But the “1k.dat” stays in the _queue folder for a very long time, until the “30g.dat” transferring to s3 server finish, then the dispatcher start dispatching the file “1k.dat” and remove it from _queue folder.
- Similarly upload “2k.dat” to Artifactory, notice the same behavior as “1k.dat”.
- Do a thread dump after uploading “2k.dat”( “30g.dat” transferring is still ongoing) , found 5 s3 worker threads, but only one thread is “Runnable”, all others are “Parking”. Attached my captured thread dump.
Binarystore.xml file used: