SQLite Forum

VFS extensions for compression & web access
Login
Dear all,
I'd like to show & tell a couple of VFS extensions I've been hacking on:

- [zstd_vfs](https://github.com/mlin/sqlite_zstd_vfs/) for read/write storage compression, similar in principle to ZIPVFS (but probably much worse)
- [web_vfs](https://github.com/mlin/sqlite_web_vfs) for read-only access to database files by http/https URL

I wrap these both in the [Genomics Extension](https://github.com/mlin/GenomicSQLite), along with a bunch of domain-specific features, with nicely-packaged language bindings. That's the easiest way to try them out, as the bioinformatics stuff can be ignored without harm.

My aim has been to make SQLite more applicable in a variety of "big data" settings, where now one sticks json.gz files on cloud storage (or other formats typically associated with JVM stacks).

OTOH I wouldn't recommend these tools for critical OLTP applications, as they're purely "use at your own risk" and not generally up to SQLite's quality/testing/portability standards. (I don't have time to press them there myself, but would welcome collaborators!)

-Mike Lin