SQLite Forum

Multi-threaded non-serialization
Login
I figured I had better ask before I goof up my project too badly. I have run into an interesting problem relating to concurrency in a multi-threaded application and I wanted to make sure I am not missing a key detail.

Let me illustrate my setup: I have multiple threads running concurrently performing simple SELECTs on a single table.

If I config for multi-threaded, setup one connection per thread as read-only with no mutex, and the default private cache, I see something like this as a result.

|
|----|__(query exec)__|  
|
|----|__(query exec)__________________|  
|
|----|__(query exec)__________________|  
|

The queries all begin concurrently, but only the first thread completes in the time that I expect it to. All subsequent threads/connections take orders of magnitude longer to complete (eg. expecting ~250 us, and subsequent threads are ~2-3 ms)

After pouring over the docs, especially everything related to isolation, I enabled shared cache and enabled "PRAGMA read_uncommitted". (still running multi-threaded, one connection per thread, with no mutex)

|
|----|__(query exec)__|  
|
|---------------------|__(query exec)__|  
|
|--------------------------------------|__(query exec)__|  
|

Now the query execution time is short as expected and the same across threads, 
however it is serialized. Reading through the docs:
"sharedcache -> 2.3. Schema (sqlite_schema) Level Locking", I would guess that I am hitting the schema read-lock and I am not able to turn that off even though I am performing read-only operations exclusively during this time.

So finally my question: Is their any mode, combination of options, compile-time variable that will disable the schema lock and allow non-serialized multi-thread execution? Especially since I can guarantee that all queries are read-only during this time.

Bonus question: Why when using a private cache do subsequent threads have such a large increase in execution time? I assume, without really digging through code, that the private cache is being rebuilt for some reason on those threads. Could I determine this, and possibly force each thread to build cache once?

I appreciate any insight.