There is a VM with a minio 120 TB, 60 CPU, 100 GB RAM. About 240 million files in one folder, the nesting is not large, 2-3 folders. The Minio cli does not work, or rather it coped with the amount of data when there were about 100 million files. While the system is counting files, all timeouts are passing. The only thing that works is to build a list of files and then in 2-3 days.
How can I transfer data from this large and heavy VM to a minio cluster?
I tried using the minio cli – the system cannot count files
transfer files via list building and rsync – yes, it is possible
Wel is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.