SQLite Forum

Database corruption: database disk image is malformed
Login

Database corruption: database disk image is malformed

(1) By wubo (wushifu) on 2021-07-12 06:27:12 [link] [source]

When query the view of the database, the following error message may occurs:

769617-view_v1_contact_methods:
769618:Error: database disk image is malformed

We have got many cases from our customers nowadays. When I run PRAGMA integrity_check to check the corruption database file, I got the following message:

*** in database main ***
Page 372 is never used
row 15 missing from index raw_contact_sort_key1_index
row 16 missing from index raw_contact_sort_key1_index
row 17 missing from index raw_contact_sort_key1_index
row 37 missing from index raw_contact_sort_key1_index
row 39 missing from index raw_contact_sort_key1_index
row 41 missing from index raw_contact_sort_key1_index
row 60 missing from index raw_contact_sort_key2_index
row 72 missing from index raw_contact_sort_key1_index
row 95 missing from index raw_contact_sort_key1_index
row 99 missing from index raw_contact_sort_key1_index
After I run the reindex command, the database seems repaired and no error message occured.
I try to reproduce the corruption issue. I use multiple threads to update the database and at the same time crash the system. But the issue still not reproduced.

(2) By Larry Brasfield (larrybr) on 2021-07-12 07:21:35 in reply to 1 [link] [source]

I do not see a question here, but I assume you want to know what to do about this ongoing issue.

My recommendation is that you look at these 3 articles, How To Corrupt Your Database Files, SQLite Over a Network, and How To Corrupt An SQLite Database File, then determine which of the hazardous practices cautioned against therein match something your application or your customers may be doing. If there are any matches, you need to either prevent those things from being done or become resigned to seeing this sort of trouble.

This forum is a good place to discuss alternatives to the discouraged practices in those article.

(3) By wubo (wushifu) on 2021-07-12 07:21:39 in reply to 1 [link] [source]

The paltform is Android 10, and linux version is 4.14.141+
I have read the article How To Corrupt An SQLite Database File.
But I'm not familiar with database or sqlite. I suspect the sync failure or filesystem corruption. But I can't reproduce this issue.

(4) By Simon Slavin (slavin) on 2021-07-12 14:05:46 in reply to 3 [source]

You've described the setup in some detail. This makes diagnosis more easy. Thank you.

With literally billions of installations, and a lot of expertise here in this forum, we don't know of any bugs which cause systematic corruption at the low level you report. I would be surprised if you can find a way to make SQLite corrupt its own database in a way that triggers that error message.

Given that you're using Android, you're probably using a mobile device. (Yes, Android runs on TVs and such things, but that's a tiny proportion of platforms.). Mobile devices can lose power in the middle of sync() operations. They can also terminate processes and threads without warning. I suspect that to reproduce the errors your clients are seeing you would have to operate your devices in low-battery situations, or to have incoming phonecalls when the device is low on free memory. Without those, since I'm sure your software has no bugs in it, you will get no corruption.

Perhaps the most useful questions to ask your customers are things like "What was the last operation you did with the App before it reported that corruption ?" "After doing that operation, was the device shut down properly, or did it run out of power, or did you have to force-quit Apps, or was there an incoming phonecall ?".

One pattern of programming I've seen on mobile devices is that rather than keep a database connection open all the time an App is running, the App closes it when it's not expecting any database use. So, for example, if the user is on the homescreen, or choosing an operation, the database is closed. When the user picks a menu option for "List entries" or "Make a new entry" a connection is opened. If the connection is not used for 60 seconds, the App closes it.