cksumvfs producing invalid checksums
(1.1) By Andrew Chambers (achamb) on 2021-07-27 07:08:34 edited from 1.0 [source]
I have been attempting to add page checksums to an sqlite3 database used by my backup software, but have found when using https://www.sqlite.org/cksumvfs.html I am unable to migrate an existing database without getting checksum failures.
Below is a git repository containing the database and a script to reproduce the issue:
https://github.com/andrewchambers/sqlitecksumvfsreport
A summary of the problem:
cp backup.sendlog backup.copy.sendlog
# Vacuum works fine initially, integrity checks pass.
./sqlite3-shell <<EOF
.open backup.copy.sendlog
vacuum;
vacuum;
pragma integrity_check;
EOF
# Enable cksumvfs following sqlite website.
# Second vacuum fails with failed checksum, even though first was ok.
./sqlite3-shell <<EOF
.load ./cksumvfs.so
.open backup.copy.sendlog
.filectrl reserve_bytes 8
vacuum;
vacuum;
EOF
produces the output:
+ cp backup.sendlog backup.copy.sendlog
+ ./sqlite3-shell
ok
+ ./sqlite3-shell
8
Error: near line 5: disk I/O error
Could anyone help me confirm if this is a bug, or if I am doing anything wrong?
Thanks for any help.
(2) By Andrew Chambers (achamb) on 2021-07-27 02:13:48 in reply to 1.0 [link] [source]
As a follow up, I can seemingly work around the issue by doing:
./sqlite3-shell <<EOF
.load ./cksumvfs.so
.open backup.copy.sendlog
.filectrl reserve_bytes 8
vacuum;
pragma checksum_verification=OFF;
vacuum;
pragma checksum_verification=ON;
vacuum;
EOF
Though the fact we need to do this pragma dance still seems a bit strange to me.
(3) By Andrew Chambers (achamb) on 2021-08-11 11:53:02 in reply to 1.1 [link] [source]
Anyone able to reproduce this?
(4) By Tyke (tyke007) on 2021-08-17 06:56:07 in reply to 3 [link] [source]
Today I clone your repo and test on zfs, get the same error.
(5) By Andrew Chambers (achamb) on 2021-10-26 23:12:20 in reply to 4 [link] [source]
Thanks for confirming.