SQLite Forum

cksumvfs producing invalid checksums
Login

cksumvfs producing invalid checksums

(1.1) By Andrew Chambers (achamb) on 2021-07-27 07:08:34 edited from 1.0 [link] [source]

I have been attempting to add page checksums to an sqlite3 database used by my backup software, but have found when using https://www.sqlite.org/cksumvfs.html I am unable to migrate an existing database without getting checksum failures.

Below is a git repository containing the database and a script to reproduce the issue:

https://github.com/andrewchambers/sqlitecksumvfsreport

A summary of the problem:

cp backup.sendlog backup.copy.sendlog

# Vacuum works fine initially, integrity checks pass.
./sqlite3-shell <<EOF
.open backup.copy.sendlog
vacuum;
vacuum;
pragma integrity_check;
EOF

# Enable cksumvfs following sqlite website.
# Second vacuum fails with failed checksum, even though first was ok.
./sqlite3-shell <<EOF
.load ./cksumvfs.so
.open backup.copy.sendlog
.filectrl reserve_bytes 8
vacuum;
vacuum;
EOF

produces the output:

+ cp backup.sendlog backup.copy.sendlog
+ ./sqlite3-shell
ok
+ ./sqlite3-shell
8
Error: near line 5: disk I/O error

Could anyone help me confirm if this is a bug, or if I am doing anything wrong?

Thanks for any help.

(2) By Andrew Chambers (achamb) on 2021-07-27 02:13:48 in reply to 1.0 [link] [source]

As a follow up, I can seemingly work around the issue by doing:

./sqlite3-shell <<EOF
.load ./cksumvfs.so
.open backup.copy.sendlog
.filectrl reserve_bytes 8
vacuum;
pragma checksum_verification=OFF;
vacuum;
pragma checksum_verification=ON;
vacuum;
EOF

Though the fact we need to do this pragma dance still seems a bit strange to me.

(3) By Andrew Chambers (achamb) on 2021-08-11 11:53:02 in reply to 1.1 [link] [source]

Anyone able to reproduce this?

(4) By Tyke (tyke007) on 2021-08-17 06:56:07 in reply to 3 [source]

Today I clone your repo and test on zfs, get the same error.

(5) By Andrew Chambers (achamb) on 2021-10-26 23:12:20 in reply to 4 [link] [source]

Thanks for confirming.