Multi-threading sluggishess on Mac?
(1) By PazO (JohnSmith) on 2021-01-13 14:09:13 [link] [source]
I had cases where a small-transaction executed from multi-threads took relatively long time.
This happens to me only on Mac.
Same code on Windows is working very fast.
Environment:
- sqlite version: 3.28.0
- OS: Mac High Sierra 10.13.6
- IDE: xcode Version 10.0 (10A255)
- SQLite configuration:
- Multi-threading mode (SQLITE_THREADSAFE==2).
- One connection per-thread.
- WAL-journal with synchronous set to NORMAL.
I noticed that while performing transaction from multiple-threads, sqlite enters sleep from the following stack:
sleep ()
unixSleep ()
sqlite3OsSleep ()
walTryBeginRead ()
sqlite3WalBeginReadTransaction ()
pagerBeginReadTransaction ()
sqlite3PagerSharedLock ()
lockBtree ()
sqlite3BtreeBeginTrans ()
sqlite3VdbeExec ()
sqlite3Step ()
sqlite3_step ()
sqlite3_exec ("BEGIN IMMEDIATE TRANSACTION;")
What I noticed is that in these cases, and while halting the debugger, I could have few threads (3~5) in sleep while no other thread is active in sqlite code.
This is possible because the sleep is not an efficient-sleep, but rather full-sleep for hard-coded period of time (several micro seconds?).
This can cause a situation where after the blocking-thread did release database-lock, the other threads are not working yet as they are still asleep.
Sleep location in sqlite code:
static int walTryBeginRead(...){
...other code...
if( cnt>5 ){
int nDelay = 1; /* Pause time in microseconds */
if( cnt>100 ){
VVA_ONLY( pWal->lockError = 1; )
return SQLITE_PROTOCOL;
}
if( cnt>=10 ) nDelay = (cnt-9)*(cnt-9)*39;
sqlite3OsSleep(pWal->pVfs, nDelay); <--- THE SLEEP
}
My question:
Can I modify something in sqlite build flags in order to make these sleeps less often?
Thanks, PazO
(2) By Dan Kennedy (dan) on 2021-01-13 15:25:15 in reply to 1 [link] [source]
Is HAVE_USLEEP defined at build time?
Dan.
(3) By PazO (JohnSmith) on 2021-01-13 16:59:22 in reply to 2 [link] [source]
Not that I am aware of.
Is this a flag that I can define at build time?
What does it do?
Thanks, PazO
(4) By Larry Brasfield (LarryBrasfield) on 2021-01-13 18:33:04 in reply to 3 [link] [source]
It causes compilation of code that can do very short sleeps instead of the longer ones available on virtually all Unixen.
(5) By PazO (JohnSmith) on 2021-01-14 06:19:02 in reply to 2 [source]
...after some testings with this flag:
Wow! what a performance boost!
Thanks very much for this advice.
PazO