I am no fool and I know exactly what caused this issue and how to make it not pop up, by not using the command that caused it, but I would rather like to use the command, and have yet to find a workaround.
The problem is simple, and it is absolutely not about file paths longer than 255 characters and the like, and "LongPathsEnabled"
is set to 1 in my [HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlFileSystem]
key.
I didn’t set it, it was already set. The error has nothing to do with long filename and the text is misleading:
FileNotFoundError: [WinError 206] The filename or extension is too long
The problem is simple, I have downloaded 424 archives from www.nexusmods.com programmatically, and I am trying to extract them automatically, these archives have redundant nested folders and way too many junk files that I don’t want to extract.
I have written efficient code to extract files in the way I like asynchronously in Python without using 7z.exe. But for .7z files unfortunately I find 7z.exe to be the only effective tool for extracting .7z files.
However 7zip lacks official Python bindings and the only way to control exactly which files are extracted to where is passing command line arguments.
I use this syntax to extract selected files only:
7z e -aoa "C:blah barfoo.7z" "-oD:a b c" "-i!a b c1" "-i!a b c2" "-i!a b c3"
I have to add one -i!
switch for every one of the files I want to include, which will make the command to be way too long, and I expected FileNotFoundError
will be triggered, so I tested it, and indeed it was triggered.
I programmatically found the archive containing the most files, which has 9571 files in it.
I constructed a command in the following way, then subprocess.run
it:
cmd = ["7z", "e", "-aoa", longname, "-o"+longname[:-4]] + ["-i!" + name for name in list(arar.paths)]
And the aforementioned error is triggered.
I have determined the exact point of failure,
cmd = ["7z", "e", "-aoa", longname, "-o"+longname[:-4]] + ["-i!" + name for name in list(arar.paths)[:388]]
Fails, but the following doesn’t:
cmd = ["7z", "e", "-aoa", longname, "-o"+longname[:-4]] + ["-i!" + name for name in list(arar.paths)[:387]]
In [39]: cmd = ["7z", "e", "-aoa", longname, "-o"+longname[:-4]] + ["-i!" + name for name in list(arar.paths)[:388]]
In [40]: len("".join(cmd))
Out[40]: 31612
In [41]: len(" ".join(cmd))
Out[41]: 32004
In [42]: cmd = ["7z", "e", "-aoa", longname, "-o"+longname[:-4]] + ["-i!" + name for name in list(arar.paths)[:387]]
In [43]: len(" ".join(cmd))
Out[43]: 31919
In [44]: len("".join(cmd))
Out[44]: 31528
It seems that the command line is limited to 32000 characters per command…
I know I can just include whole folders and exclude some files and include all files of certain extensions… to optimize the command line length, but this is just too much work for little gain.
Plus I now know it is more efficient to extract whole 7z archives than extracting files individually, so I will just extract everything and then delete all the trash afterwards.
But this makes me wonder if there is a workaround to the character limit of commands, maybe I one day will actually need a command longer than 32000 characters, say for machine learning or artificial intelligence?
So is there any workaround?