SQLite Forum

after crash loosing all records in a multi-GB table
Login
Hmm, now I am getting a bit confused: 

We *are* actually committing (END TRANSACTION) every 5 seconds. We are exclusively writing to one data base connection from a single thread without interspersed concurrent reads, so everything is as sequential and controlled 
as it can be. I am also quite certain that the commit is happening, as I  remember that during development without the BEGIN/END TRANSACTION we have just seen growing RAM consumption and no on disk file growth.

Thus, my expectation really *was* that we would at most loose the last 5 seconds worth of data. The reason for my post was that we collected about 70 GB (about 500..1000 seconds) of data on disk (=final database file size) at the moment of the crash, but observed that, after the journal was rolled back, the file was truncated to about 500 kB. 

The entire code is too involved to be posted, but if the behavior may indicate a bug in sqlite and anybody is interested, I could probably undertake the endeavour to produce a test case...