SQLite User Forum

Auto-vacuuming and concurrency
Login

Auto-vacuuming and concurrency

(1) By anonymous on 2022-05-13 04:29:07 [link] [source]

Hello, I have a couple of questions about auto-vacuuming.

First, with auto_vacuum=FULL, the docs say that, after every transaction, SQLite will try and shrink the database by moving free pages to the end and then truncating. That sounds like it will take a write lock (and, indeed, I see in the code that doing a manual incremental_vacuum is a write operation), even after read-only transactions, which is less than ideal for concurrency.

If my understanding of this is correct, I think it should be noted in the docs for auto_vacuum and incremental_vacuum, so that you don't have to be making deductions to avoid performance traps.

Second, how does auto-vacuuming work with WAL mode? It can't just move pages around and truncate the database immediately, because there might be readers concurrently looking at the database. So the auto-vacuum needs to go through the WAL, which seems sensible for moving the free pages around, but it's not totally immediately clear when the truncation can happen - it must be after a checkpoint completes, but how does SQLite know to do it then, which is at a remove from the auto-vacuum (or incremental vacuum) operation? Does checkpointing entail truncating free pages at the end of the database if auto-vacuuming is enabled (even in incremental mode)? If so, I think that's also worth noting in the docs, though as an explanation for the curious rather than a notice for the unwary.

(2) By Richard Hipp (drh) on 2022-05-13 13:52:16 in reply to 1 [source]

The auto-vacuum only happens on a write transaction. For a read transaction, nothing has changed, and so there is nothing to auto-vacuum. Hence the write lock is already held whenever an auto-vacuum occurs.

In WAL mode, the database changes necessary to move freelist pages to the end of the database and then truncate the database are written into the WAL file. But those changes are not actually applied (and the database does not actually shrink) until the next checkpoint operation is run to move the changes in the WAL file back into the database.