I have 100s of CSVs that total to 150+ GB of data. I want to merge them all into one CSV file, so I can then dump that CSV file into an SQLite database for easier querying.
This stackoverflow post (R – How to combine many .csv files into one without overloading memory?) suggested the csvstack utility to do so, with the ability of just specifying the folder itself and a wildcard filename.
However, this does not seem to work on Windows – I get the error OSError: [Errno 22] Invalid argument: '*.csv'
. Specifying each file name individually is not an option.
How can I get the wildcard selection working on windows?
I’ve tried running the utility from inside the folder and just specifying *.csv
as input, as well as from outside the folder and specifying folder/*.csv
as input.
AKnum is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.