SQLite Forum

SQLite_Master data out of nowhere
Login

SQLite_Master data out of nowhere

(1) By anonymous on 2021-12-02 13:24:30 [link] [source]

Hello all.

We are using System.Data.SQLite on WinCE13. Suddenly on one of our devices in our setting table the first 31 entries were overwritten with the ones from SQLite_Master. The SQLite_Master is not used directly anywhere in our code and none of us had worked manually on the database.

We are using both vacuum and auto vacuum and the device could perhaps have been turned off by power loss. We suspect there are problems there, however the journal file should prevent such issues.

Do you guys have any ideas on how such data could get into another table out of nowhere?

Thanks in advance

(2) By Simon Slavin (slavin) on 2021-12-02 17:07:39 in reply to 1 [link] [source]

Sorry, I have only bad news.

This isn't a commonly-found problem. The normal cause of these things is that something in your code is stomping on memory SQLite itself uses: a pointer gets the wrong value, or the wrong part of memory is overwritten. The cause can be a bug in your code, a bug in a library you're calling, a bug in your OS, or a hardware glitch. I'm not saying that there's 0% chance that it's a bug in SQLite, merely that I don't remember anyone else mentioning that problem.

Perhaps someone else can think of a reason why SQLite might want to read sqlite_master after the database has been opened. I can't. Mmmm … unless you do a VACUUM in your code.

(3) By anonymous on 2021-12-03 08:17:25 in reply to 2 [link] [source]

First thanks for the quick answer.

Yes, we automatically use VACUUM in our code after every 10th start of the device. Before the VACUUM we have a SELECT to get the counter and after the VACUUM we INSER/UPDATE the new counter value. The counter is in the setting table and with SELECT/UPDATE the database is already open so it could really be the reason.

A different question about VACUUM. Should we full avoid automatically VACUUM or is it enough if we separate the VACUUM with a restart from any other SQL command?

(4) By Simon Slavin (slavin) on 2021-12-03 17:07:45 in reply to 3 [source]

What you're doing should not cause the behaviour you're seeing. But perhaps it does explain why SQLite does have the sqlite_master table handy when it wants to corrupt your file. It should not be possible to corrupt your database by using VACUUM, and if you stop using VACUUM it probably just means that your data-corruption bug remains, and it'll overwrite your database with something else instead.

There's nothing wrong with VACUUM. It has no reputation for being buggy. But in a production setting it's normally used only for special occasions like a quarterly maintenance routine. SQLite automatically reuses any space saved by deleting data, and the large amount of time VACUUM takes more than makes up for time saved during normal operations on all but the slowest rotating hard disk. In addition, it involves a lot of operations for SSDs and Flash memory, reducing the lifetime of such devices.

The only time I suggest VACUUM used in normal operation is if you are copying/transferring a copy of your database and you want it to be as small as possible so it can be copied/transferred as quickly as possible.

Was your problem a one-off event, or does it happen more often than that ?