SQLite Forum

Multi-threaded non-serialization
Login
After much head banging, I found it! Thanks to all for the hints, it is a File lock issue.

This is from the amalgamation: in the function "winLock" (since I am in a windows environment) about line 44954 there is this bit of code
```
/* Lock the PENDING_LOCK byte if we need to acquire a PENDING lock or
  ** a SHARED lock.  If we are acquiring a SHARED lock, the acquisition of
  ** the PENDING_LOCK byte is temporary.
  */
  newLocktype = pFile->locktype;
  if( pFile->locktype==NO_LOCK
   || (locktype==EXCLUSIVE_LOCK && pFile->locktype<=RESERVED_LOCK)
  ){
    int cnt = 3;
    while( cnt-->0 && (res = winLockFile(&pFile->h, SQLITE_LOCKFILE_FLAGS,
                                         PENDING_BYTE, 0, 1, 0))==0 ){
      /* Try 3 times to get the pending lock.  This is needed to work
      ** around problems caused by indexing and/or anti-virus software on
      ** Windows systems.
      ** If you are using this code as a model for alternative VFSes, do not
      ** copy this retry logic.  It is a hack intended for Windows only.
      */
      lastErrno = osGetLastError();
      OSTRACE(("LOCK-PENDING-FAIL file=%p, count=%d, result=%d\n",
               pFile->h, cnt, res));
      if( lastErrno==ERROR_INVALID_HANDLE ){
        pFile->lastErrno = lastErrno;
        rc = SQLITE_IOERR_LOCK;
        OSTRACE(("LOCK-FAIL file=%p, count=%d, rc=%s\n",
                 pFile->h, cnt, sqlite3ErrName(rc)));
        return rc;
      }
      if( cnt ) sqlite3_win32_sleep(1);
    }
```
If two or more threads attempt to set the "PENDING" bit at the same time, obviously all but the first will fail. That in turn hits that sleep call at the end, which is where the consistent 1-2ms timing issue was showing up.

So.... I would challenge the design of this area. Why, when we only want to obtain a shared lock are we setting the pending bit, then clearing it a few steps later??

For my purposes, I am changing the condition to: 
`if( locktype==EXCLUSIVE_LOCK && pFile->locktype<=RESERVED_LOCK )`  
That way, ONLY when I am intending to write to the file, will I request a pending lock. It works for me. I am by no means an expert on this library, but I would put a little more thought into the design here, or at the very least decrease the sleep into sub-millisecond timing.