SQLite WASM benchmarks
(1) By mlaw (tantaman) on 2023-03-14 13:34:17 [link] [source]
Any progress on getting benchmarks such as those here: https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html for the WASM build?
I'm looking for benchmarks on how long it takes to send a query from the UI thread to the worker thread and get that data all the way back to the UI thread when using the OPFS VFS.
Query performance only inside the worker isn't of that much interest given, ultimately, you need to get the data back to the UI.
Also, have any further explorations been done on using WebLocks for better concurrency?
(2) By Stephan Beal (stephan) on 2023-03-14 13:59:10 in reply to 1 [source]
Any progress on getting benchmarks such as those here: https://rhashimoto.github.io/wa-sqlite/demo/benchmarks.html for the WASM build?
Nope. i'm currently "between residences" and nice-to-haves, except for the most low-lying of fruit, currently have to wait until i've found a place to live and get moved in. There's unfortunately still no estimate on how long that's going to take (ongoing since January).
We placed very tight performance requirements on ourselves going into this, benchmarked constantly during the OPFS VFS's development, and bent over backwards to eke out every bit of performance we could, eventually getting it down to approximately 3x native C speed (give or take a bit, depending on the tests in question). That is to say: it's not slow.
Passing data between threads will slow things down, of course, but that overhead is unavoidable and not something we can influence. As a basis for comparison, our tests late last year measured an average of approximately 40% of the OPFS VFS's runtime being spent waiting on cross-thread communication between the VFS's synchronous and async halves. All VFS-level data transfer is performed via a SharedArrayBuffer, so has no extraneous cross-thread copy costs - that performance hit is us waiting on the browser's thread synchronization and communication.
Also, have any further explorations been done on using WebLocks for better concurrency?
Nope, for the same reason. The current OPFS concurrency solution seems to work reasonably well, so WebLocks experimentation is way down the list of current priorities.
(3) By Daniel Steigerwald (steida) on 2023-03-14 16:54:19 in reply to 2 [link] [source]
Off-topic: Don't buy a house yet; rent it. Meanwhile, watch The Big Short (film). It does not explain properly what happened (it completely ignores QE), but it's fun to watch, and it describes well what will happen soon. TLDR; Houses will be much cheaper.
(4.1) By mlaw (tantaman) on 2023-03-15 17:04:51 edited from 4.0 in reply to 2 [link] [source]
I've started putting some benchmarks together and I have to say I'm impressed with how far things have come since I was last using the official SQLite WASM build.
Concurrency looks like it still has a long way to go, however.
The benchmarks are not ready for public consumption (they likely only make sense to me right now) but will be available here: https://github.com/tantaman/sqlite-wasm-bench
(5) By Stephan Beal (stephan) on 2023-03-15 18:13:06 in reply to 4.1 [link] [source]
Concurrency looks like it still has a long way to go, however.
We'll never have desktop-grade concurrency in the browser, don't aim to, and i'll likely continue to politely harangue users who insist on requiring that in their web apps ;).
Getting any concurrency at all, because of OPFS's exclusive-locking semantics, requires actively fighting against OPFS in some way or other because OPFS only directly supports exclusive locks. We are assured that the folks responsible for creating the APIs are working on extensions for POSIX-esque locking, which would integrate far more fluidly with C-style APIs like ours, but there's no word on when those might become available. When they do, we're eager to try them out.
We have a very basic application for testing the concurrency at the bottom of:
https://wasm-testing.sqlite.org/
(and in the source tree under ext/wasm/tests/opfs/concurrency, reachable via ext/wasm/index.html)
with instructions at the top for how to change its parameters.
By and large, three connections can run reliably on the same db concurrently, provided they are careful not to hold locks for any significant length of time. (To be honest, three seems "perfectly reasonable" to me for this use case, given the target environment and OPFS's locking limitations.) Each connection after that increases the odds of locking errors considerably. Adding the "unlock-asap" flag roughly doubles the concurrency capabilities but at a significant performance hit (roughly 400% performance hits were measured in I/O-intensive tests). i've seen as many as 10 connections run contention-free with the unlock-asap mode in that app, but honestly suspect that that was a bit of a fluke. Five or six seems (via lots of highly unscientific testing) like safe upper limit for unlock-asap mode.
More info on the unlock-asap mode can be found at wasm:/doc/trunk/persistence.md#vfs-locking.