Intent behind xSleep() and whether it can be a no-op
(1) By jaskij on 2022-09-29 19:50:55 [link] [source]
What is the intent behind xSleep()
in sqlite3_vfs
? For context, I am porting SQLite to a system without meaningful sleep implementation (microcontroller without an RTOS) and am left with two options:
- Make
xSleep()
a no-op - Wait in a busy loop until the required time has passed, with millisecond resolution
Looking through the code further, it seems that sleep is mainly used for backoff, in cases where multiple threads or processes are trying to access the database file simultaneously. This will never be the case in our particular use.
(2) By Warren Young (wyoung) on 2022-09-29 20:21:42 in reply to 1 [link] [source]
What is the intent behind
xSleep()
To implement this, I assume.
This will never be the case
How can you be so certain?
Take the case of a data logger. You may indeed have only a single-threaded program running without an RTOS writing to the database, as you say, but how does the user get the data out? Who reads the data and presents it to the user? Options:
Stop the writer and use the same thread to export the data. You've maintained single-accessor status, so you could not only stub out the sleep function, you could probably put a null pointer here and not cause a crash.
Single writer, but the user is allowed to copy the file out using some filesystem primitives, web access, etc. You've probably corrupted the database by not using the backup API or another method that does the locking properly.
Single writer, parallel reader to batch download updates. Unless you're running in WAL mode, you probably do need a busy handler. Whether that's SQLite's own or a hand-rolled loop around the code handling the
SQLITE_BUSY
return code is up to you.
(3) By jaskij on 2022-09-29 20:43:25 in reply to 2 [link] [source]
How can you be so certain?
First of all, this is more a using SQLite as an application format use case. So, while there will be writes, they will be sporadic, in response to the user changing something via the interface. Although data logging is possible in the future.
The underlying filesystem implementation, to the best of my knowledge, does not support locking files, so accessing the database from multiple places would be a programming error, which will be communicated to developers writing the downstream code.
As to how the user accesses the database, without going through the device's interface, it will most likely be through either using our device as USB mass storage or through physically taking out the SD card. You raise good points about corruption, so I will impress upon the people responsible to include it in the UI and manual. Thank you for this.
(4) By Stephan Beal (stephan) on 2022-09-29 20:44:23 in reply to 1 [link] [source]
What is the intent behind xSleep() in sqlite3_vfs? For context, I am porting SQLite to a system without meaningful sleep implementation (microcontroller without an RTOS) and am left with two options:
FWIW, i recently asked this question to Richard in the context of a JavaScript sqlite3_vfs implementation, where sleep was apparently impossible to implement, and he said something along the lines of, "so it won't sleep." i.e. a no-op is, for certain contexts, fine. Sleeping is used while waiting on locks, and if you can't have locks in your environment, or won't wait on them, then a no-op is fine.
(We ended up finding a synchronous implementation of xSleep() in JS, but it requires the SharedArrayBuffer and Atomics APIs, both of which are disabled unless specific security-related response headers are sent with the script by the web server.)
(5) By jaskij on 2022-09-29 21:07:10 in reply to 4 [link] [source]
I could do blocking/sync xSleep()
, but it does not really make sense to me since:
- It blocks the execution of the entire processor (barring interrupts)
- I can't have locks
Thank you for the reply.
(6) By ddevienne on 2022-09-30 09:40:01 in reply to 4 [source]
unless specific security-related response headers are sent with the script by the web server
Ah, that explains https://sqlite.org/althttpd/info/85b7ee71fa19264317f7 then. Good to know.
(7) By Stephan Beal (stephan) on 2022-09-30 10:45:38 in reply to 6 [link] [source]
Ah, that explains https://sqlite.org/althttpd/info/85b7ee71fa19264317f7 then.
Indeed. That option was not added specifically to support the JS-side xSleep() but to enable other cross-thread synchronization in the sqlite3 JS API, the gory details of which are buried in the opfs-proxy-atomic branch. The ability to implement xSleep() on top of that came along as a side effect.