I’m trying to process a bunch of files (several thousand, each with a size of around 500KB) with a C++ executable, but I’m experiencing very poor performance when reading the files sequentially. The core C++ code is basically this:
int main() {
std::vector<std::string> filePaths{ "file1", "file2", ... }; // Many file paths
int i = 0;
for (const std::string& filePath : filePaths) {
std::ifstream file(filePath, std::ios::binary);
std::cout << i++ << std::endl;
}
return 0;
}
I compiled this with MSVC 2022 as well as MinGW (GCC 13.1.0), both with standard CMake Release optimizations on. However the runtime performance on my quite modern machine (Win11, 32GB RAM, Ryzen 9, SSD) is far beneath of what I would have expected:
Extracting the std::ifstream file
object out of the loop and using file.open
/file.close
in the loop or even using C-style fopen
/fclose
instead of std::ifstream
makes no difference.
Am I missing something? I can’t believe that it takes several minutes to just open (not even process yet) a few thousand small files…