SQLite Forum

Writing a SQLite db as separate files

Writing a SQLite db as separate files

(1) By anonymous on 2021-09-15 11:22:24 [link] [source]

I would like to store a SQLite database as separate files (for example, in 512K chunks), automatically creating a new file when the last one exceeds the max file size. I'm guessing this would require a custom VFS. Is anyone aware of a VFS already written to do this?

The reason I want this is so I can continuously back up a SQLite database to a cloud object store such as S3, which normally don't allow partial updates to files (they must be fully replaced). If I keep it in a single file, any small change will cause the entire file to be re-uploaded. Keeping the database as separate chunks would minimize how much I would need to upload after making a small change to the database.

(2) By Larry Brasfield (larrybr) on 2021-09-15 11:38:18 in reply to 1 [source]

You might consider using rsync.

(3) By Richard Hipp (drh) on 2021-09-15 11:39:33 in reply to 1 [link] [source]