SQLite Forum

Additional .001 .002 .. huge files get created sporadically
Login

Additional .001 .002 .. huge files get created sporadically

(1) By supermario (marcbauchet) on 2021-12-06 10:18:11 [link] [source]

Using the SQLite C/C++ API (devkit version 3.36.0), my_database.db sometimes suddenly grows to almost 2 GB (2,097,088 KB) and a my_database.001 gets created, up to the same size, and sometimes also a my_database.002, etc. These files are not valid SQLIte files or anything I can makes sense of.

Additional information:

  • We're using PRAGMAs synchronous=FULL, journal_mode=OFF, locking_mode=EXCLUSIVE, temp_store=2.
  • We have to set these pragmas because we don't want SQLite to create any extra file.
  • With the SQLite API we mostly do lots of INSERTs, REPLACEs and DELETEs, on a single table, and my_database.db is typically 1.5GB or less.

Questions:

  • This is sporadic, so reproducible case have been very elusive. Would anyone have a suggestion which way to look?
  • On a couple occasions HAS happened that the database got corrupt (possibly because of crashes during development or because Visual Studio halted the program during a write operation) but the behavior was different, we could just see it with a PRAGMA quick_check. Could database corruption also cause this behavior?

(2) By Dan Kennedy (dan) on 2021-12-06 10:51:36 in reply to 1 [source]

You have somehow ended up using a VFS named "multiplex", designed for use with file-systems that do not support files larger than 2GiB:

https://sqlite.org/src/file?name=src/test_multiplex.c&ci=trunk

If you're not using this VFS intentionally, the solution is to find and remove the call to sqlite3_multiplex_initialize() in your code.

Dan.

(3) By supermario (marcbauchet) on 2021-12-06 15:07:01 in reply to 2 [link] [source]

right on, this is exactly the size SQLITE_MULTIPLEX_CHUNK_SIZE 2147418112 and the code that generate those file names, thanks!