Troubleshooting corrupt indexes
(1) By Jim Borden (borrrden) on 2020-04-17 00:05:06 [source]
Posting this on behalf of a colleague who "doggedly denies the existence of the new forum":
We've found a few database instances with index corruption, and I'm unsure how to troubleshoot it.
The symptom is SELECT queries failing with SQLITE_CORRUPT
. I've looked at the database files, and pragma integrity_check
spits out five "row missing from index" errors on one index, and "wrong # of entries" on that same index and three others. Running REINDEX
seems to repair everything, at least integrity_check reports no more problems.
The indexes aren't fancy. The one with the row-missing errors is simply
CREATE UNIQUE INDEX seq ON table (sequence)
where the sequence
column is simply defined as sequence INTEGER
.
The other indexes are all of the form
CREATE INDEX … ON table (customfn(body, 'key1'), customfn(body, 'key2'), …)
where customfn
is a custom function that extracts key-value data from the structured blob in body
, somewhat like the regular json_extract
function.
I'm sure this custom function could conceivably cause this sort of error if it weren't properly deterministic, but it's been in use for a few years and pretty well tested, and I've never seen a bug with index corruption like this.
And I'm baffled how something as simple as the seq
index could go awry…
We're using SQLite 3.28, statically linked into the app, on Android.
(2) By Warren Young (wyoung) on 2020-04-17 00:48:28 in reply to 1 [link] [source]
I'm sure this custom function could conceivably cause this sort of error if it weren't properly deterministic
The function has to be more than simply deterministic. It has to obey all of the following restrictions lined out in the docs:
- If A==B then B==A.
- If A==B and B==C then A==C.
- If A<B THEN B>A.
- If A<B and B<C then A<C.
If your collation function doesn't obey them all, corruption is possible.