WebAssembly, a.k.a. WASM, is a standard defining a low-level programming language suitable (A) as a target for cross-compilation from many other languages and (B) for running via a virtual machine in a browser. Designed with scriptability via JavaScript in mind, it provides a way to compile C code (among others) to WASM and script it via JavaScript with relatively little friction despite the vast differences between JavaScript and C.

Folks have been building sqlite3 for the web since as far back as 2012 but this subproject is the first effort "officially" associated with the SQLite project, created with the goal of making WASM builds of the library first-class members of the family of supported SQLite deliverables.

Specific Goals of this Project

The concrete goals of this project include...

• Except where noted in the non-goals, provide a more-or-less feature-complete wrapper to the sqlite3 C API, insofar as WASM feature parity with C allows for. In fact, provide at least the following APIs...

  1. Bind a low-level sqlite3 API which is as close to the native one as feasible in terms of usage.
  2. A higher-level OO API, more akin to sql.js and node.js-style implementations. This one speaks directly to the low-level API. This API must be used from the same thread as the low-level API.
  3. A Worker-based API which speaks to the previous APIs via Worker messages. This one is intended for use in the main thread, with the lower-level APIs installed in a Worker thread, and talking to them via Worker messages. Because Workers are asynchronous and have only a single message channel, some acrobatics are needed here to feed async work results back to the client (as we cannot simply pass around callbacks between the main and Worker threads).
  4. A Promise-based variant of the Worker API (#3, above) which entirely hides the cross-thread communication aspects from the user.

• Insofar as possible, support persistent client-side storage using available JS APIs. As of this writing, that includes the Origin-Private FileSystem (OPFS) and (very limited) storage via the window.localStorage and window.sessionStorage backend.

Specific Non-goals

Things we specifically do not aim to achieve:

• As WASM is a web-centric technology and UTF-8 is the King of Encodings in that realm, there are no current plans to support the UTF16-related sqlite3 APIs. They would add a complication to the bindings for no appreciable benefit.

• Though support for out-of-browser WASM runtimes is widespread, this project is currently focused only on browser targets. Though web-related implementation details take priority, and the JavaScript components of the API specifically focus on browser clients, the lower-level WASM module "should" work in non-web WASM environments, assuming that the environment can provide the "imports" which the WASM file needs.

• Supporting old or niche-market platforms. WASM is built for a modern web and requires modern platforms. Similarly, sqlite3 library options which have been deprecated are not included in the WASM interface.

Attribution

Several projects have helped us considerably along the way. We are greatly indebted to (in the order we investigated them)...

Emscripten

https://emscripten.org

The Emscripten WASM toolchain is, as of this writing, the only full-featured WASM toolchain available. Though other toolchains exist, Emscripten offers several "killer features" which others do not, most notably transparent emulation of POSIX file I/O APIs, which allows sqlite3 to function as-is in a WASM build, with no changes or tweaking necessary.

Additionally, Emscripten developers directly offered invaluable support during the development of the OPFS-based features.

sql.js

https://github.com/sql-js/sql.js

Alon Zakai's sql.js was an essential stepping stone in this code's development as it demonstrates how to handle some of the WASM-related voodoo (like handling pointers-to-pointers and adding JS implementations of C-bound callback functions). These APIs have a considerably different shape than sql.js, however.

As far as we're aware, sql.js was the first-ever published use of sqlite3 for the web.

absurd-sql

https://github.com/jlongster/absurd-sql

James Long's aptly-named absurd-sql demonstrates persistent browser-side sqlite3 by storing the databases in IndexedDB storage. We experimented with that approach but it didn't suit us. Even so, it was an interesting experiment.

wa-sqlite

https://github.com/rhashimoto/wa-sqlite

Roy Hashimoto's wa-sqlite was the first project to public an OPFS storage option for sqlite3. It also provides a full demonstration of how to use the otherwise largely undocumented OPFS APIs.