SQLite Forum

Multi-threading sluggishess on Mac?
Login

Multi-threading sluggishess on Mac?

(1) By PazO (JohnSmith) on 2021-01-13 14:09:13 [source]

I had cases where a small-transaction executed from multi-threads took relatively long time.
This happens to me only on Mac.
Same code on Windows is working very fast.

Environment:

  • sqlite version: 3.28.0
  • OS: Mac High Sierra 10.13.6
  • IDE: xcode Version 10.0 (10A255)
  • SQLite configuration:
    • Multi-threading mode (SQLITE_THREADSAFE==2).
    • One connection per-thread.
    • WAL-journal with synchronous set to NORMAL.

I noticed that while performing transaction from multiple-threads, sqlite enters sleep from the following stack:

  • sleep ()
  • unixSleep ()
  • sqlite3OsSleep ()
  • walTryBeginRead ()
  • sqlite3WalBeginReadTransaction ()
  • pagerBeginReadTransaction ()
  • sqlite3PagerSharedLock ()
  • lockBtree ()
  • sqlite3BtreeBeginTrans ()
  • sqlite3VdbeExec ()
  • sqlite3Step ()
  • sqlite3_step ()
  • sqlite3_exec ("BEGIN IMMEDIATE TRANSACTION;")

What I noticed is that in these cases, and while halting the debugger, I could have few threads (3~5) in sleep while no other thread is active in sqlite code.
This is possible because the sleep is not an efficient-sleep, but rather full-sleep for hard-coded period of time (several micro seconds?).
This can cause a situation where after the blocking-thread did release database-lock, the other threads are not working yet as they are still asleep.

Sleep location in sqlite code:

static int walTryBeginRead(...){

    ...other code...

    if( cnt>5 ){
        int nDelay = 1;                      /* Pause time in microseconds */
        if( cnt>100 ){
            VVA_ONLY( pWal->lockError = 1; )
            return SQLITE_PROTOCOL;
        }

        if( cnt>=10 ) nDelay = (cnt-9)*(cnt-9)*39;
        sqlite3OsSleep(pWal->pVfs, nDelay);	 <--- THE SLEEP
}

My question:
Can I modify something in sqlite build flags in order to make these sleeps less often?

Thanks, PazO

(2) By Dan Kennedy (dan) on 2021-01-13 15:25:15 in reply to 1 [link] [source]

Is HAVE_USLEEP defined at build time?

Dan.

(3) By PazO (JohnSmith) on 2021-01-13 16:59:22 in reply to 2 [link] [source]

Not that I am aware of.
Is this a flag that I can define at build time?
What does it do?

Thanks, PazO

(4) By Larry Brasfield (LarryBrasfield) on 2021-01-13 18:33:04 in reply to 3 [link] [source]

It causes compilation of code that can do very short sleeps instead of the longer ones available on virtually all Unixen.

(5) By PazO (JohnSmith) on 2021-01-14 06:19:02 in reply to 2 [link] [source]

...after some testings with this flag:

Wow! what a performance boost!
Thanks very much for this advice.

PazO