The NTFS Compression technology is transparent, that when reading and writing, the compression
and decompression are handled automatically.
This is good and convenient. However, when copying a NTFS-compressed file, that file just
firstly gets decompressed in the reading process, then gets compressed again in the writing process.
That is the case when copying a NTFS-compressed file into a folder than doesn’t automatically
compress new file.
When copying a NTFS-compressed
file into a folder that automatically compresses a new file, there is an extra
step. When writing, the file gets compressed again.
I haven’t tested, but I guess although during the copy process the file is firstly decompressed then compressed again, the
stored compressed data just gets unchanged.
That is, although the underlying compressed data doesn’t get changed, the data gets
decompressed and compressed again, which slows the copy process down, sometimes
quite significantly.
In short, consider the original
file data D
and the NTFS-compress’d
data C
. The source file is
NTFS-compress’d, so the underlying stored
data is C
. Then, during the copy
progress, the compressed data C
is firstly decompressed into memory
to be D
– probably by blocks, instead
of as a whole file – then getting compressed
again, with the compressed data C
gets written back into the disk.
The current copy process is
C -> D -> C
, with extra steps of
decompression and compression in the middle.
If the API looking for is present,
it would be simply C -> C
, straightforward
copy without decompression and compression.
Therefore, it would be good if there is an API that copies the NTFS-compressed file without decompressing and compressing again, but just simply copies the underlying compressed bytes.
However, it looks like although there is a function LZCopy
, it doesn’t do this:
Copies a source file to a destination file. If the source file has been compressed by the Lempel-Ziv algorithm, this function creates a decompressed destination file. If the source file is not compressed, this function duplicates the original file.
(https://learn.microsoft.com/en-us/windows/win32/api/lzexpand/nf-lzexpand-lzcopy)
In short, is there a Windows API that copies the NTFS-compressed file without decompressing and compressing again? Or, is there not one currently?
(Update below)
1. Current findings
According to the osr community post, it
should be possible to read the compressed
data via documented Defrag API, FSCTL,
and/or DeviceIoControl.
But it seems unmentioned
for the second part, that is, how to write
the compressed data to a file, while
directing the system that the
data to write are already compressed,
so that it writes the already compressed
data, instead of attempting to compress
the already compressed data rather again.
The post: https://community.osr.com/t/copy-ntfs-compressed-files-without-decompression/25109
2. But what for? Why should this be needed?
Recently I was copying files between some old
HDDs, and in my environment there is a significant
difference of copy speed when the destination
disk is NTFS compression enabled.
Since virtually all of the source files I am
copying are NTFS-compress’d, it occurs to me
that it would be much better if there is an API
to just copy the underlying compressed data,
instead of firstly decompressing, then compressing
again.
It is rather out of curiousity for me
than for practical use, since for now I just
temporaily turns off the ‘Compress new
files added into this folder’ in the
copy destination, and will compress the data
in the copy destination disk
later after the whole copy process is done.
10