Unfortunately I wasn't able to confirm that this commit is indeed the problem

It's the most likely culprit but i don't have a hypothesis about why that browser misbehaves there while the others don't (and don't use that platform, so have no basis for testing).

There is an alternative solution for that part which i've avoided because it's dog ugly and not 64-bit compatible, but that latter part won't be much of an issue for at least a few years¹.

i will attempt that change this evening (CET), but am tied up until then with relocation-related hassles.

as earlier commits had other unrelated problems.

Also Safari-specific?

Sidebar: all platform compatibility for WASM and JS for platforms i don't have is based off feature compatibility charts on MDN. It's a known issue that Safari older than v15 (Sept. 2021) won't work because it was missing some of the relevant BigInt pieces. There is no workaround for that other than to elide the 64-bit pieces of the build altogether, which our build process can do but we don't do in our canonical builds. That said, the problem you're reporting is unrelated to the int64-related pieces.

1. ^{^} He says naively.

There is an alternative solution for that part which i've avoided because it's dog ugly ...

That's now in the trunk. Please try it at your convenience and report any problems.

Thank you for your continued feedback!

Gave it a shot but a slightly different error is now thrown:

[Error] sqlite3 bootstrap initializer threw:
Error: Internal error: sqlite3.wasm.exports[sqlite3_free] is not the same pointer as SQLITE_WASM_DEALLOC. These must match in order to accommodate allocator-related API guarantees. — sqlite3.js:10860
	sqlite3ApiBootstrap (sqlite3.js:7827)
	(anonymous function) (sqlite3.js:17099)
	callRuntimeCallbacks (sqlite3.js:1307)
	postRun (sqlite3.js:937)
	doRun (sqlite3.js:5753)
	run (sqlite3.js:5766)
	runCaller (sqlite3.js:5707)
	removeRunDependency (sqlite3.js:1049)
	receiveInstance (sqlite3.js:1198)
	(anonymous function) (sqlite3.js:151)
	promiseReactionJob

If it helps, I created a playwright configuration. It'll let you run some automated tests across chrome, webkit & firefox (and others if you enable them) in your environment.

You can see that here: https://github.com/vlcn-io/sqlite/pull/1

Installation and management of browsers is taken care of so you won't have to worry about that.

Gave it a shot but a slightly different error is now thrown:

That's a different formulation of the same problem report. The difference in the previous checkin was how the pointer being checked gets installed from WASM into JS.

This is a bug in Safari, plain and simple. There is one more thing to try, but such acrobatics shouldn't be necessary to transfer a pointer from WASM to JS. That's now on trunk, but it's a painfully dissatisfying solution (if indeed it is one).

Please try it out at your convenience, but at this point i'm expecting the same stacktrace as you just reported. If you do see that, please post the output of the following from the dev console of the tester1.html main-thread tests:

S.wasm.exports.sqlite3_wasm_ptr_to_sqlite3_free()

It should output an integer (a pointer into the "indirect function table"). On Chrome and FF it currently emits 3, but that's a WASM-internal impl detail and subject to change in any given build so cannot be relied upon. If it does output an integer then that part works and we then need to test the second part of that constellation:

S.wasm.functionEntry(S.wasm.exports.sqlite3_wasm_ptr_to_sqlite3_free())

That "should" emit some platform-specific form of a function object. On FF it looks like:

function 79() // noting that the function's name is WASM-internal and unimportant

in Chrome/ium it looks like:

ƒ $sqlite3_free() { [native code] }

And please post the output of:

S.wasm.exports.sqlite3_free // WITHOUT PARENS - don't call it

That final output must match the output of the above functionEntry() call. If it doesn't, the browser is broken (unable to look up functions by their WASM function pointer) and it's something the vendor will have to resolve.

As a point of reference, this is what Chrome tells me, and FF says something equivalent:

> S.wasm.exports.sqlite3_wasm_ptr_to_sqlite3_free()
3
> S.wasm.functionEntry(S.wasm.exports.sqlite3_wasm_ptr_to_sqlite3_free())
ƒ $sqlite3_free() { [native code] }
> S.wasm.exports.sqlite3_free
ƒ $sqlite3_free() { [native code] }

Sidebar: that check is what's being performed by the bit which is triggering the exception you've posted.

If it helps, I created a playwright configuration.

i have no idea what that means but after looking at the link, that's not something i can spend time on until some indeterminate point after my move, which i'm still frantically packing, cleaning, and otherwise preparing for.

Installation and management of browsers is taken care of so you won't have to worry about that.

Until something doesn't work ;).

If you do see that, please post the output of the following from the dev console of the tester1.html main-thread tests:

S.wasm.exports.sqlite3_wasm_ptr_to_sqlite3_free()

Same exception and unfortunately S is undefined. Given the exception is thrown in sqlite3ApiBootstrap, it must be preventing that variable from being assigned. To double check myself, I repeated the same process in Chrome and S.wasm.exports... is defined there, so it isn't an error in your instructions or my copy-pasting.

Anyway -- I set some breakpoints to gather the information you need:

At sqlite3.js:11656:

capi.SQLITE_WASM_DEALLOC = wasm.exports.sqlite3_wasm_ptr_to_sqlite3_free();

The assigned value is 3

At sqlite3.js:11657:

if(wasm.exports[sqlite3.config.deallocExportName]
       !== wasm.functionEntry(capi.SQLITE_WASM_DEALLOC)){

A watch expression for the left hand side of !==

wasm.exports[sqlite3.config.deallocExportName]

evaluates to the function named 79

(note: sqlite3.config.deallocExportName is set to sqlite3_free)

The watch expression for the right hand side of !==

wasm.functionEntry(capi.SQLITE_WASM_DEALLOC)

evaluates to an anonymous function with no name. FWIW, all entries in wasm.exports.__indirect_function_table are anonymous functions with no name on Safari.

I'll keep doing some digging on my end.

Could Safari be doing some wrapping of the functions that are placed into the function table? Where the entry is indeed the expected function but with some wrapping on top that breaks equality?

Some additional details --

putting all function table entires in a set:

const fsets = new Set();
for (let i = 0; i < S.wasm.exports.__indirect_function_table.length; ++i) {
  fsets.add(S.wasm.exports.__indirect_function_table.get(i))
}

then filtering exports against it:

let had = Object.entries(S.wasm.exports).filter((e) => fsets.has(e[1]))

results in 3 entries on Chrome:

['sqlite3_free', ƒ]
'sqlite3changeset_old', ƒ]
'sqlite3changeset_new', ƒ]

and 0 on Safari.

putting all function table entires in a set: ... results in 3 entries on Chrome:

Coincidentally, i noticed that (in a slightly different form) in a similar experiment a few days ago. Chrome's behavior there is admittedly strange. Why only those three functions get that treatment is inexplicable. It doesn't make a difference at the level we're working at, so isn't worth exploring deeply, but it is exceptionally curious. There's certainly a story behind that somewhere.

You can see something similar by simply iterating over the indirect function table and sending each to console.log. Only those 3 functions have (in Chrome/ium) the same names as they do in C, whereas the rest all have synthesized names.

From the dev console of tester1.html:

const n = S.wasm.functionTable().length;
for(let i = 0; i < n; ++i) {
  const f = S.wasm.functionEntry(i);
  if(f) console.log(f);
}

Same exception and unfortunately S is undefined.

Good early morning. My apologies if that wasn't clear: the S symbol is only installed in the main test app (tester1.html/js), not when a client installs the module. It's there solely to simplify experimentation via the dev console.

FWIW, all entries in wasm.exports.__indirect_function_table are anonymous functions with no name on Safari.

Well that's... unfortunate. That's different than every other browser does it.

That said... (head-scratching...) hypothetically that's... possibly... okay. We can remove the sanity check which is triggering that exception for you and see what happens. That check is there because...

1. To make sure the browser is behaving as expected. In hindsight, it comes as little surprise that there's an outlier browser which behaves unexpected.

2. To ensure that SQLITE_WASM_DEALLOC holds the proper function pointer. If it doesn't then memory leaks will ensue in all sorts of operations, e.g. binding blobs and text to bound parameters.

Sidebar: the SQLITE_WASM_DEALLOC "macro" is used in the same way as the C-side SQLITE_STATIC and SQLITE_TRANSIENT function pointers - passed to certain operations as a destructor function for a pointer passed in to those same functions (most prominently sqlite3_bind_text() and sqlite3_bind_blob()). It was added in order to eliminate some extra memory-copying which otherwise needs to be done by such functions.

If you would, please: simply delete that "if" block which performs that check (the one from which that exception is thrown) and if that truly works for you then i'll hesitantly remove that sanity check altogether.

The trick, however, is ensuring that it actually works properly. The primary symptom of failure would not be an outright immediate failure but memory leaks.

Could Safari be doing some wrapping of the functions that are placed into the function table? Where the entry is indeed the expected function but with some wrapping on top that breaks equality?

That's absolutely what it sounds like. Why they do that is anyone's guess, but that's irrelevant for our purposes - simply knowing that it behaves that way means we can't rely on checks like the one you've been triggering. Shrug.

Thank you again for your continued testing and feedback!

If you would, please: simply delete that "if" block which performs that check (the one from which that exception is thrown) and if that truly works for you then i'll hesitantly remove that sanity check altogether.

That sanity check has now been replaced with a simple check of whether functionEntry(SQLITE_WASM_DEALLOC) exists, rather than trying to ensure that it's exactly the same function as sqlite3_free(). And now we know not to rely on identify comparisons for functions served from the indirect function table.

Again, thank you for making the effort to get to the bottom of this!

Thanks. Were you able to confirm that no memory leaks were introduced? I did comment out the if yesterday and things "ran fine" but I haven't gotten around to crafting a test that relies on SQLITE_WASM_DEALLOC and grabbing heap dumps before & after the test run.

Were you able to confirm that no memory leaks were introduced?

Only insofar as i am able to confirm (via the dev tools) that SQLITE_WASM_DEALLOC is indeed the function pointer it needs to be. The check you were triggering was just an automated assertion of that.

I did comment out the if yesterday and things "ran fine" but I haven't gotten around to crafting a test that relies on SQLITE_WASM_DEALLOC and grabbing heap dumps before & after the test run.

If you can confirm that SQLITE_WASM_DEALLOC is an integer and that that integer's slot is used in the indirect function table, that will suffice as a sanity check. There's an automated check for that in place, it just doesn't (any longer) verify that the function in that slot is in fact sqlite3_free. Since we're getting that pointer value from the C code, however, there's no reason to believe that it could ever be any other function (barring malicious manipulation).

i.e. pulling heap dumps shouldn't be necessary.

Ran into one more Safari oddity today when trying to spin up opfs. Safari doesn't support spawning workers from within workers so the opfs proxy can't start up :/

I see that the bug was fixed in WebKit in September of 2022 but curious if you have (or know someone who has) any insider knowledge on when this fix would make it into a Safari release.

Ran into one more Safari oddity today when trying to spin up opfs. Safari doesn't support spawning workers from within workers so the opfs proxy can't start up :/

That's a limitation we cannot currently support :(, just like we can't support loading an ES6 module from a Worker in Firefox until they add that capability.

I see that the bug was fixed in WebKit in September of 2022 but curious if you have (or know someone who has) any insider knowledge on when this fix would make it into a Safari release.

Unfortunately no. My recollection (very possibly incorrect!) from my only meeting with a few Safari folks late last summer was that they were missing several of the sync-related OPFS APIs. Since Chromium is our baseline for All Things OPFS, i've not made any concentrated effort to keep track OPFS support in the other browsers.

Apple is good about acknowledging a bug, and very bad about saying when it will fix it, or when the fix will reach public releases.

You might try the current Technology Preview version of Safari, which gets fixes before the official release. It's free and you don't need an account or anything.

https://developer.apple.com/safari/technology-preview/

This has the added advantage of an integrated bug-reporting facility: if you report a bug it can package up the page you're looking up and several context things which will let Apple figure out what's going on, and send them with your report. It does not do this unless you tell it to, and it tells you what it's doing.

"Safari doesn't support spawning workers from within workers"

There is a polyfill that I am using Evolu https://github.com/johanholmerin/nested-worker The approach is sound.

That's a good (and somewhat hilarious) polyfill, thanks for posting.

It has to be modified to work in current webkit, but it's possible to get it running without errors in latest Safari Technogy Preview.

Unfortunately another webkit bug is preventing sqlite form actually working at the moment: https://bugs.webkit.org/show_bug.cgi?id=250495. It's fixed, but hasn't made it into TP yet. Thought I post this here in case someone else is debugging this.

If you're trying to use SQLite with the OPFS backend and it fails, this post may be relevant.