The JS code of SQLite-WASM makes heavy use of self which does not work in Node.js.

We don't target node js and currently have no timeline for when we might do so.

As window.self is specified in the HTML standard, all browsers support it.

That's currently our only target.

The JS code provided with the wasm build is very specifically browser-only. It would tickle me pink to see node users create node-compliant variants of it, but it is not currently planned within this project for the very simple reason that none of us use node in any capacity.

The JS community has recognized the need for defining a reference to the global scope independent of the environment and has settled on globalThis:

That's new info for me, thank you.

It is more recent than self, so older browsers might not support it - Chrome got it in 2018. However if one compares it to the WASM/SharedArrayBuffer support - switching from self to globalThis won't break any compatible browsers:

SAB is only needed for OPFS support and as-yet-hypothetical similar future APIs and the canonical builds require BigInt and BigInt64Array, which Safari only got in late 2020 resp. 2021. (It's possible to build without BigInt support, but that will eliminate much of the SQLite API.)

i.e. there are no real compatibility concerns if it goes back to around 2018/19. Both MDN and caniuse agree that its adoption is well within "the safe zone" in terms of compatibility.

i will get us switched to globalThis, but we still don't specifically target node.js ;).

Ok, I got it working in Node.js. There were only very minor incompatibilities, mostly related to window.location.

Here is the list of the changes I made:

1. In GNUMakefile:

   emcc.environment := -sENVIRONMENT=web,worker,node
   

   (this adds a Node.js runtime, including a readBinary() that loads the WASM blob from the filesystem instead of calling fetch)

2. Disabled this sanity check:

   if('function' !== typeof importScripts){
   toss("initWorker1API() must be run from a Worker thread.");
   }
   

3. Changed a number of these:

   new URL(self.location.href).searchParams
   

   into

   self.location?.href ? new URL(self.location.href).searchParams : new URLSearchParams()
   

   (in Node.js everything is loaded directly from the filesystem)

4. Changed

   self.location.pathname.endsWith('.worker.js')
   

   to

   self.location?.pathname?.endsWith?.('.worker.js')
   

The truth is that there already is a very good SQLite Node.js addon based on the classical C-API that will surely have higher performance than the WASM version, but there are a number of cases where the WASM version might be preferable - unit testing of web components and React-Native / NativeScript come to mind.

In GNUMakefile:

No problem.

Disabled this sanity check:

We need a check which forces that code to fail loudly if it's run in an non-Worker environment. Is your objection to the check itself or to the way the check is done?

If it's the former, then "tough luck!" :) If it's the latter then there are better/cleaner/more precise ways to check that which i didn't know at the time that was written, so it can be easily changed... except that i'd need one detail from you: a JS snippet which can unambiguously tell us whether we're running in node. In essence we need to distinguish between 3 environments: main browser thread, browser worker thread, and "something else." i've got the first two covered, but not the last one.

self.location?.href ? new URL(self.location.href).searchParams : new URLSearchParams()

We can do that (with globalThis instead).

self.location?.pathname?.endsWith?.('.worker.js')

That part probably isn't needed anymore - it was required for wasmfs support, which subsequently stripped out due to incompatible changes they made late last year. i'll go through that bit and see if it can be removed altogether.

i'll get to these tomorrow at the latest and get back to you. Thanks again for the feedback.

Here is the list of the changes I made:

Except for:

Disabled this sanity check:

those changes are now in the trunk and a the prerelease snapshot has been updated.

That other check was changed to:

if(!(globalThis.WorkerGlobalScope instanceof Function)){
  toss("initWorker1API() must be run from a Worker thread.");
}

The full diffs, compared to a day ago: src:/vdiff?from=46b3ac6d1fdd9207&to=2f712b836a0dafd0

If it needs additional adjustments, please let me know. Thank you for your continued feedback.

Alas, this new build does not work with webpack. The Node.js code that Emscripten injects breaks webpack bundling. I have found an old issue on Github, but there is no solution at the moment.

This is what emscripten uses to detect Node.js:

var ENVIRONMENT_IS_NODE = typeof process == 'object' && typeof process.versions == 'object' && typeof process.versions.node == 'string';

Then it proceeds to import 'module':

if (ENVIRONMENT_IS_NODE) {
  const { createRequire } = await import('module');

webpack compilation uses AST parsing and it is not smart enough to correctly eliminate this import.

It can do:

if (false) {
  const { createRequire } = await import('module');

But even:

const ENVIRONMENT_IS_NODE = false;
if (ENVIRONMENT_IS_NODE) {
  const { createRequire } = await import('module');

As you already know, JS is a jungle with no rules. I maintain a number of NPM modules and supporting both CJS and ES6 is already a hassle, add browser/Node.js and it becomes even worse, add WASM and Workers, and it becomes an absolute nightmare.

I think that with the current system, there is no other choice besides supporting Node.js with a separate bundle, one built with -sENVIRONMENT=node when the day for Node.js comes.

webpack#7352

I finally have something working using this snapshot. Nothing too interesting, but in-memory setup worked.

The README has the details:

https://github.com/mandolyte/sqlitenext