I have 3 millions small files(12-15kb each) in S3 directory. I want to download 20000 files in each chunk from S3. Is there any way where I can download multiple files at once by limiting file numbers?
I tried listObjectsV2Request using multi-threading as well, but trying to find out better way to improve performance.
Also I tried using transferManager.downloadDirectory, but which is downloading complete directory causing outOfMemoryError.