SQLite User Forum

Calling fsync() does not necessarily ensure that the entry in the directory containing the file has also reached disk
Login

Calling fsync() does not necessarily ensure that the entry in the directory containing the file has also reached disk

(1.1) By mosolov on 2024-07-03 08:20:54 edited from 1.0 [link] [source]

I have a question about fsync, as of man ( https://man7.org/linux/man-pages/man2/fsync.2.html in the description section):

Calling fsync() does not necessarily ensure that the entry in the directory containing the file has also reached disk. For that an explicit fsync() on a file descriptor for the directory is also needed.
https://github.com/sqlite/sqlite/blob/3d24637325188c1ed9db46e5bb23ab5d747ad29f/src/os_unix.c#L3634

^^^^^^^

Can someone please explain me why there's no fsync on a file dir as man recommends?

XY problem: The issue is I have vfat fs on MicroSD on ARM+Embedded Linux (Kernel 3.10). My app does fsync on settings file, it's just regular binary data of different size depending on count of startup commands, e.g. write(&C_struct, ..., N*commands_size). Common scenario: user changes settings (just a file on MicroSD vfat) of device startup procedure (app ack settings write after fsync of settings file so data makes it to actual storage I suppose :D ), waiting ~1 minute and then user cuts off power from device to check startup procedure and there's a chance that settings file truncates to size 0 for some reason.

I've changed the code to (simplified, drop all error checks):

void fsync_wrap(FILE *f, const char *filedir_path) { int fd = fileno(f); fsync(fd); // <--- fsync on file descriptor

DIR *dir = opendir(filedir_path);
int dir_fd = dirfd(dir);
retval = fsync(dir_fd);     // <--- fsync on file dir
closedir(dir);    

}

But I have doubts does it fix the issue or no. I've seen some weird (for me) mentions of MicroSD card can have it's own internal cache of data to write to actual storage so it might report to the upper level data is written meanwhile data is not written to the actual storage and powerloss = dataloss.

Actually I'm very interested in an advice about how to debug that issue, e.g. virtualize SoC by QEMU, automate the reproduce of the issue e.g. make a tear setup with setting drop power N msec after fsync and try to get bingo msec value to reproduce the issue by 100% rate.

(2) By Simon Slavin (slavin) on 2024-07-03 12:52:48 in reply to 1.1 [link] [source]

I don't know how the software works, but I can confirm how the driver works.

What speed is your MicroSD storage ? It goes by class, which you should find printed on the label. Class 4 >= 4MB/s, Class 6 >= 6MB/s, etc.. If you add protocol overheads to that, you halve the speed. So changing 2 MB of a SQLite database can take a full second on a Class 4 card. If your SQLite program did this properly it would freeze, allowing no processing, for a full second every time it wrote to the database.

Because they run so slowly, the Windows/macOS/Linux drivers for most MicroSD read/write units lie. There are no circumstances under which they hold up the controlling program until the entire writing process is complete. Changes are accumulated in a cache, the API call returns, and changes are written to the card in the background while the CPU continues with other things.

There's an additional problem: SD cards wear out proportional to the number of write operations. So to make the cards last longer, drivers will batch up write operations until they fill the cache, then write them all in one operation or until they've had no additional write calls for a few seconds. So no, you cannot depend on cards being up-to-date unless you've chosen 'eject'. The OS has no clue that this is happening. It's not the OS's problem.

If your OS insists on warning-before-dismount, for example requiring you to choose 'eject' before saying it's safe to take the card out, it won't allow it until the write-cache is empty. Which is good. But, as you write above, you may still get corruption through hardware failure or power loss. You can see the effects of this if you tell a camera to shut down and sometimes there's a puzzling long pause before you can see the power actually go off.

From a data-safety point of view (which I used to have professionally) we don't consider SD storage to be ACID-compliant. Sorry about that. They're fine if you don't need that.

(4) By mosolov on 2024-07-04 05:32:28 in reply to 2 [source]

What speed is your MicroSD storage ?

Class 10, actually the size of writing is ~10 kB.

SD cards wear out proportional to the number of write operations I thought SD card controller manages that issue, unlike "dumb" memory like NAND which requires some SW component like UBI FS to address this issue.

The problem is it's industrial device that can loss power any time. I think it's design flaw and Micro SD + FAT fs just don't fit to this requirements.

Thanks for your time!

(6) By Simon Slavin (slavin) on 2024-07-04 12:22:27 in reply to 4 [link] [source]

Okay, paying for Class 10 suggests you're doing serious work with serious hardware. So you probably approached everything else in a serious manner.

DRH's response about the internals of SQLite suggests that SQLite itself (and therefore your programming) is doing everything it can to ensure that your device is both ACID and up-to-date.

So I share you conclusions. The only thing I can think of is to run something in the background which tells the card "You're about to be ejected" once every few seconds. That should force the SD firmware to be sure that the card is updated. But this is such an obvious thought that I'm suspecting it's an option in the driver configuration.

(7) By Richard Hipp (drh) on 2024-07-04 12:34:55 in reply to 6 [link] [source]

I have seen Linux kernel builds for which fsync() on a USB drive was a no-op. I dunno if that is the case for your setup or not. I only observer that it has happened before.

(3) By Richard Hipp (drh) on 2024-07-03 13:45:30 in reply to 1.1 [link] [source]

Can someone please explain me why there's no fsync on a file dir

I believe your assumption is incorrect. SQLite does fsync the directory containing the file. See the code at https://sqlite.org/src/info/2ea8d3ed496b8d1f933?ln=3801-3803

(5) By mosolov on 2024-07-04 05:33:14 in reply to 3 [link] [source]

Yep, I was wrong, thank you very much for pointing that block of code!