I’m working on a database project where I need to write to both a data.bin file (for storing data) and a metadata file. I’m running into an issue with the overhead caused by filesystem metadata updates, especially when using undo logs. Every write operation essentially doubles, leading to 8x writes for a single data update (2 writes for data and meta multiplied by 2 writes for undo logs, multiplied by 2 due to filesystem).

To reduce this overhead, I’m considering bypassing the filesystem for metadata management and directly managing sector writes myself. My idea is to:

Preallocate a large block of sectors for the data.bin file.
Maintain a pointer in my metadata file, which I would advance (like fseek) each time I allocate a new sector.
The challenge I’m facing is that I can’t find a reliable way to get all the extensions (start addresses, sectors forming the file) that the filesystem allocates to my data.bin file. Without this information, I can’t properly manage the sectors directly and avoid filesystem metadata updates every time I allocate a new sector.

Is there a way to retrieve the specific sector addresses or clusters that a file occupies on disk? Or, is there a better approach to achieve what I’m trying to do, which is to minimize the number of writes required for data and metadata updates by bypassing the filesystem’s metadata management?

Total Writes with Undo file system updates and undo logs:

Undo Log Writes (2x): Writing original data and metadata to the undo log.
Data and Metadata Writes (2x): Performing the actual data write and updating the filesystem metadata.
Filesystem Writes (2x): Updating the filesystem with the new undo log entries and final metadata.
Thus, you have:

2 Writes: Undo log (original data + metadata).
2 Writes: Actual data and metadata updates.
2 Writes: Updating the filesystem with the undo log entries.
2 Writes: Updating the filesystem with the final metadata.

I’d like to eliminate the filesystem metadata update at each write since I want to provide my own metadata for my db anyway and use the drivers that my mcu provides to drive directly to my sd card sectors.. Thanks for any help, greatly appreciated (PS: I use fatfs)