Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Added ext/async/sqlite3async.c.

            1  +/*
            2  +** 2005 December 14
            3  +**
            4  +** The author disclaims copyright to this source code.  In place of
            5  +** a legal notice, here is a blessing:
            6  +**
            7  +**    May you do good and not evil.
            8  +**    May you find forgiveness for yourself and forgive others.
            9  +**    May you share freely, never taking more than you give.
           10  +**
           11  +*************************************************************************
           12  +**
           13  +** $Id: sqlite3async.c,v 1.1 2009/04/23 14:58:40 danielk1977 Exp $
           14  +**
           15  +** This file contains an example implementation of an asynchronous IO 
           16  +** backend for SQLite.
           17  +**
           18  +** WHAT IS ASYNCHRONOUS I/O?
           19  +**
           20  +** With asynchronous I/O, write requests are handled by a separate thread
           21  +** running in the background.  This means that the thread that initiates
           22  +** a database write does not have to wait for (sometimes slow) disk I/O
           23  +** to occur.  The write seems to happen very quickly, though in reality
           24  +** it is happening at its usual slow pace in the background.
           25  +**
           26  +** Asynchronous I/O appears to give better responsiveness, but at a price.
           27  +** You lose the Durable property.  With the default I/O backend of SQLite,
           28  +** once a write completes, you know that the information you wrote is
           29  +** safely on disk.  With the asynchronous I/O, this is not the case.  If
           30  +** your program crashes or if a power loss occurs after the database
           31  +** write but before the asynchronous write thread has completed, then the
           32  +** database change might never make it to disk and the next user of the
           33  +** database might not see your change.
           34  +**
           35  +** You lose Durability with asynchronous I/O, but you still retain the
           36  +** other parts of ACID:  Atomic,  Consistent, and Isolated.  Many
           37  +** appliations get along fine without the Durablity.
           38  +**
           39  +** HOW IT WORKS
           40  +**
           41  +** Asynchronous I/O works by creating a special SQLite "vfs" structure
           42  +** and registering it with sqlite3_vfs_register(). When files opened via 
           43  +** this vfs are written to (using sqlite3OsWrite()), the data is not 
           44  +** written directly to disk, but is placed in the "write-queue" to be
           45  +** handled by the background thread.
           46  +**
           47  +** When files opened with the asynchronous vfs are read from 
           48  +** (using sqlite3OsRead()), the data is read from the file on 
           49  +** disk and the write-queue, so that from the point of view of
           50  +** the vfs reader the OsWrite() appears to have already completed.
           51  +**
           52  +** The special vfs is registered (and unregistered) by calls to 
           53  +** function asyncEnable() (see below).
           54  +**
           55  +** LIMITATIONS
           56  +**
           57  +** This demonstration code is deliberately kept simple in order to keep
           58  +** the main ideas clear and easy to understand.  Real applications that
           59  +** want to do asynchronous I/O might want to add additional capabilities.
           60  +** For example, in this demonstration if writes are happening at a steady
           61  +** stream that exceeds the I/O capability of the background writer thread,
           62  +** the queue of pending write operations will grow without bound until we
           63  +** run out of memory.  Users of this technique may want to keep track of
           64  +** the quantity of pending writes and stop accepting new write requests
           65  +** when the buffer gets to be too big.
           66  +**
           67  +** LOCKING + CONCURRENCY
           68  +**
           69  +** Multiple connections from within a single process that use this
           70  +** implementation of asynchronous IO may access a single database
           71  +** file concurrently. From the point of view of the user, if all
           72  +** connections are from within a single process, there is no difference
           73  +** between the concurrency offered by "normal" SQLite and SQLite
           74  +** using the asynchronous backend.
           75  +**
           76  +** If connections from within multiple processes may access the
           77  +** database file, the ENABLE_FILE_LOCKING symbol (see below) must be
           78  +** defined. If it is not defined, then no locks are established on 
           79  +** the database file. In this case, if multiple processes access 
           80  +** the database file, corruption will quickly result.
           81  +**
           82  +** If ENABLE_FILE_LOCKING is defined (the default), then connections 
           83  +** from within multiple processes may access a single database file 
           84  +** without risking corruption. However concurrency is reduced as
           85  +** follows:
           86  +**
           87  +**   * When a connection using asynchronous IO begins a database
           88  +**     transaction, the database is locked immediately. However the
           89  +**     lock is not released until after all relevant operations
           90  +**     in the write-queue have been flushed to disk. This means
           91  +**     (for example) that the database may remain locked for some 
           92  +**     time after a "COMMIT" or "ROLLBACK" is issued.
           93  +**
           94  +**   * If an application using asynchronous IO executes transactions
           95  +**     in quick succession, other database users may be effectively
           96  +**     locked out of the database. This is because when a BEGIN
           97  +**     is executed, a database lock is established immediately. But
           98  +**     when the corresponding COMMIT or ROLLBACK occurs, the lock
           99  +**     is not released until the relevant part of the write-queue 
          100  +**     has been flushed through. As a result, if a COMMIT is followed
          101  +**     by a BEGIN before the write-queue is flushed through, the database 
          102  +**     is never unlocked,preventing other processes from accessing 
          103  +**     the database.
          104  +**
          105  +** Defining ENABLE_FILE_LOCKING when using an NFS or other remote 
          106  +** file-system may slow things down, as synchronous round-trips to the 
          107  +** server may be required to establish database file locks.
          108  +*/
          109  +
          110  +#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ASYNCIO)
          111  +
          112  +#include "sqlite3async.h"
          113  +
          114  +#define ENABLE_FILE_LOCKING
          115  +
          116  +#ifndef SQLITE_AMALGAMATION
          117  +# include "sqliteInt.h"
          118  +# include <assert.h>
          119  +# include <string.h>
          120  +#endif
          121  +
          122  +/* Useful macros used in several places */
          123  +#define MIN(x,y) ((x)<(y)?(x):(y))
          124  +#define MAX(x,y) ((x)>(y)?(x):(y))
          125  +
          126  +/* Forward references */
          127  +typedef struct AsyncWrite AsyncWrite;
          128  +typedef struct AsyncFile AsyncFile;
          129  +typedef struct AsyncFileData AsyncFileData;
          130  +typedef struct AsyncFileLock AsyncFileLock;
          131  +typedef struct AsyncLock AsyncLock;
          132  +
          133  +/* Enable for debugging */
          134  +static int sqlite3async_trace = 0;
          135  +# define ASYNC_TRACE(X) if( sqlite3async_trace ) asyncTrace X
          136  +static void asyncTrace(const char *zFormat, ...){
          137  +  char *z;
          138  +  va_list ap;
          139  +  va_start(ap, zFormat);
          140  +  z = sqlite3_vmprintf(zFormat, ap);
          141  +  va_end(ap);
          142  +  fprintf(stderr, "[%d] %s", 0 /* (int)pthread_self() */, z);
          143  +  sqlite3_free(z);
          144  +}
          145  +
          146  +/*
          147  +** THREAD SAFETY NOTES
          148  +**
          149  +** Basic rules:
          150  +**
          151  +**     * Both read and write access to the global write-op queue must be 
          152  +**       protected by the async.queueMutex. As are the async.ioError and
          153  +**       async.nFile variables.
          154  +**
          155  +**     * The async.pLock list and all AsyncLock and AsyncFileLock
          156  +**       structures must be protected by the async.lockMutex mutex.
          157  +**
          158  +**     * The file handles from the underlying system are not assumed to 
          159  +**       be thread safe.
          160  +**
          161  +**     * See the last two paragraphs under "The Writer Thread" for
          162  +**       an assumption to do with file-handle synchronization by the Os.
          163  +**
          164  +** Deadlock prevention:
          165  +**
          166  +**     There are three mutex used by the system: the "writer" mutex, 
          167  +**     the "queue" mutex and the "lock" mutex. Rules are:
          168  +**
          169  +**     * It is illegal to block on the writer mutex when any other mutex
          170  +**       are held, and 
          171  +**
          172  +**     * It is illegal to block on the queue mutex when the lock mutex
          173  +**       is held.
          174  +**
          175  +**     i.e. mutex's must be grabbed in the order "writer", "queue", "lock".
          176  +**
          177  +** File system operations (invoked by SQLite thread):
          178  +**
          179  +**     xOpen
          180  +**     xDelete
          181  +**     xFileExists
          182  +**
          183  +** File handle operations (invoked by SQLite thread):
          184  +**
          185  +**         asyncWrite, asyncClose, asyncTruncate, asyncSync 
          186  +**    
          187  +**     The operations above add an entry to the global write-op list. They
          188  +**     prepare the entry, acquire the async.queueMutex momentarily while
          189  +**     list pointers are  manipulated to insert the new entry, then release
          190  +**     the mutex and signal the writer thread to wake up in case it happens
          191  +**     to be asleep.
          192  +**
          193  +**    
          194  +**         asyncRead, asyncFileSize.
          195  +**
          196  +**     Read operations. Both of these read from both the underlying file
          197  +**     first then adjust their result based on pending writes in the 
          198  +**     write-op queue.   So async.queueMutex is held for the duration
          199  +**     of these operations to prevent other threads from changing the
          200  +**     queue in mid operation.
          201  +**    
          202  +**
          203  +**         asyncLock, asyncUnlock, asyncCheckReservedLock
          204  +**    
          205  +**     These primitives implement in-process locking using a hash table
          206  +**     on the file name.  Files are locked correctly for connections coming
          207  +**     from the same process.  But other processes cannot see these locks
          208  +**     and will therefore not honor them.
          209  +**
          210  +**
          211  +** The writer thread:
          212  +**
          213  +**     The async.writerMutex is used to make sure only there is only
          214  +**     a single writer thread running at a time.
          215  +**
          216  +**     Inside the writer thread is a loop that works like this:
          217  +**
          218  +**         WHILE (write-op list is not empty)
          219  +**             Do IO operation at head of write-op list
          220  +**             Remove entry from head of write-op list
          221  +**         END WHILE
          222  +**
          223  +**     The async.queueMutex is always held during the <write-op list is 
          224  +**     not empty> test, and when the entry is removed from the head
          225  +**     of the write-op list. Sometimes it is held for the interim
          226  +**     period (while the IO is performed), and sometimes it is
          227  +**     relinquished. It is relinquished if (a) the IO op is an
          228  +**     ASYNC_CLOSE or (b) when the file handle was opened, two of
          229  +**     the underlying systems handles were opened on the same
          230  +**     file-system entry.
          231  +**
          232  +**     If condition (b) above is true, then one file-handle 
          233  +**     (AsyncFile.pBaseRead) is used exclusively by sqlite threads to read the
          234  +**     file, the other (AsyncFile.pBaseWrite) by sqlite3_async_flush() 
          235  +**     threads to perform write() operations. This means that read 
          236  +**     operations are not blocked by asynchronous writes (although 
          237  +**     asynchronous writes may still be blocked by reads).
          238  +**
          239  +**     This assumes that the OS keeps two handles open on the same file
          240  +**     properly in sync. That is, any read operation that starts after a
          241  +**     write operation on the same file system entry has completed returns
          242  +**     data consistent with the write. We also assume that if one thread 
          243  +**     reads a file while another is writing it all bytes other than the
          244  +**     ones actually being written contain valid data.
          245  +**
          246  +**     If the above assumptions are not true, set the preprocessor symbol
          247  +**     SQLITE_ASYNC_TWO_FILEHANDLES to 0.
          248  +*/
          249  +
          250  +
          251  +#ifndef NDEBUG
          252  +# define TESTONLY( X ) X
          253  +#else
          254  +# define TESTONLY( X )
          255  +#endif
          256  +
          257  +/*
          258  +** There are two definitions of the following functions. One for pthreads
          259  +** compatible systems and one for Win32. These functions isolate the OS
          260  +** specific code required by each platform.
          261  +**
          262  +** The system uses three mutexes and a single condition variable. To
          263  +** block on a mutex, async_mutex_enter() is called. The parameter passed
          264  +** to async_mutex_enter(), which must be one of ASYNC_MUTEX_LOCK,
          265  +** ASYNC_MUTEX_QUEUE or ASYNC_MUTEX_WRITER, identifies which of the three
          266  +** mutexes to lock. Similarly, to unlock a mutex, async_mutex_leave() is
          267  +** called with a parameter identifying the mutex being unlocked. Mutexes
          268  +** are not recursive - it is an error to call async_mutex_enter() to
          269  +** lock a mutex that is already locked, or to call async_mutex_leave()
          270  +** to unlock a mutex that is not currently locked.
          271  +**
          272  +** The async_cond_wait() and async_cond_signal() functions are modelled
          273  +** on the pthreads functions with similar names. The first parameter to
          274  +** both functions is always ASYNC_COND_QUEUE. When async_cond_wait()
          275  +** is called the mutex identified by the second parameter must be held.
          276  +** The mutex is unlocked, and the calling thread simultaneously begins 
          277  +** waiting for the condition variable to be signalled by another thread.
          278  +** After another thread signals the condition variable, the calling
          279  +** thread stops waiting, locks mutex eMutex and returns. The 
          280  +** async_cond_signal() function is used to signal the condition variable. 
          281  +** It is assumed that the mutex used by the thread calling async_cond_wait() 
          282  +** is held by the caller of async_cond_signal() (otherwise there would be 
          283  +** a race condition).
          284  +**
          285  +** It is guaranteed that no other thread will call async_cond_wait() when
          286  +** there is already a thread waiting on the condition variable.
          287  +**
          288  +** The async_sched_yield() function is called to suggest to the operating
          289  +** system that it would be a good time to shift the current thread off the
          290  +** CPU. The system will still work if this function is not implemented
          291  +** (it is not currently implemented for win32), but it might be marginally
          292  +** more efficient if it is.
          293  +*/
          294  +static void async_mutex_enter(int eMutex);
          295  +static void async_mutex_leave(int eMutex);
          296  +static void async_cond_wait(int eCond, int eMutex);
          297  +static void async_cond_signal(int eCond);
          298  +static void async_sched_yield(void);
          299  +
          300  +/*
          301  +** There are also two definitions of the following. async_os_initialize()
          302  +** is called when the asynchronous VFS is first installed, and os_shutdown()
          303  +** is called when it is uninstalled (from within sqlite3async_shutdown()).
          304  +**
          305  +** For pthreads builds, both of these functions are no-ops. For win32,
          306  +** they provide an opportunity to initialize and finalize the required
          307  +** mutex and condition variables.
          308  +**
          309  +** If async_os_initialize() returns other than zero, then the initialization
          310  +** fails and SQLITE_ERROR is returned to the user.
          311  +*/
          312  +static int async_os_initialize(void);
          313  +static void async_os_shutdown(void);
          314  +
          315  +/* Values for use as the 'eMutex' argument of the above functions. The
          316  +** integer values assigned to these constants are important for assert()
          317  +** statements that verify that mutexes are locked in the correct order.
          318  +** Specifically, it is unsafe to try to lock mutex N while holding a lock 
          319  +** on mutex M if (M<=N).
          320  +*/
          321  +#define ASYNC_MUTEX_LOCK    0
          322  +#define ASYNC_MUTEX_QUEUE   1
          323  +#define ASYNC_MUTEX_WRITER  2
          324  +
          325  +/* Values for use as the 'eCond' argument of the above functions. */
          326  +#define ASYNC_COND_QUEUE    0
          327  +
          328  +/*************************************************************************
          329  +** Start of OS specific code.
          330  +*/
          331  +#if SQLITE_OS_WIN || defined(_WIN32) || defined(WIN32) || defined(__CYGWIN__) || defined(__MINGW32__) || defined(__BORLANDC__)
          332  +
          333  +/* The following block contains the win32 specific code. */
          334  +
          335  +#define mutex_held(X) (GetCurrentThreadId()==primitives.aHolder[X])
          336  +
          337  +static struct AsyncPrimitives {
          338  +  int isInit;
          339  +  DWORD aHolder[3];
          340  +  CRITICAL_SECTION aMutex[3];
          341  +  HANDLE aCond[1];
          342  +} primitives = { 0 };
          343  +
          344  +static int async_os_initialize(void){
          345  +  if( !primitives.isInit ){
          346  +    primitives.aCond[0] = CreateEvent(NULL, TRUE, FALSE, 0);
          347  +    if( primitives.aCond[0]==NULL ){
          348  +      return 1;
          349  +    }
          350  +    InitializeCriticalSection(&primitives.aMutex[0]);
          351  +    InitializeCriticalSection(&primitives.aMutex[1]);
          352  +    InitializeCriticalSection(&primitives.aMutex[2]);
          353  +    primitives.isInit = 1;
          354  +  }
          355  +  return 0;
          356  +}
          357  +static void async_os_shutdown(void){
          358  +  if( primitives.isInit ){
          359  +    DeleteCriticalSection(&primitives.aMutex[0]);
          360  +    DeleteCriticalSection(&primitives.aMutex[1]);
          361  +    DeleteCriticalSection(&primitives.aMutex[2]);
          362  +    CloseHandle(primitives.aCond[0]);
          363  +    primitives.isInit = 0;
          364  +  }
          365  +}
          366  +
          367  +/* The following block contains the Win32 specific code. */
          368  +static void async_mutex_enter(int eMutex){
          369  +  assert( eMutex==0 || eMutex==1 || eMutex==2 );
          370  +  assert( eMutex!=2 || (!mutex_held(0) && !mutex_held(1) && !mutex_held(2)) );
          371  +  assert( eMutex!=1 || (!mutex_held(0) && !mutex_held(1)) );
          372  +  assert( eMutex!=0 || (!mutex_held(0)) );
          373  +  EnterCriticalSection(&primitives.aMutex[eMutex]);
          374  +  TESTONLY( primitives.aHolder[eMutex] = GetCurrentThreadId(); )
          375  +}
          376  +static void async_mutex_leave(int eMutex){
          377  +  assert( eMutex==0 || eMutex==1 || eMutex==2 );
          378  +  assert( mutex_held(eMutex) );
          379  +  TESTONLY( primitives.aHolder[eMutex] = 0; )
          380  +  LeaveCriticalSection(&primitives.aMutex[eMutex]);
          381  +}
          382  +static void async_cond_wait(int eCond, int eMutex){
          383  +  ResetEvent(primitives.aCond[eCond]);
          384  +  async_mutex_leave(eMutex);
          385  +  WaitForSingleObject(primitives.aCond[eCond], INFINITE);
          386  +  async_mutex_enter(eMutex);
          387  +}
          388  +static void async_cond_signal(int eCond){
          389  +  assert( mutex_held(ASYNC_MUTEX_QUEUE) );
          390  +  SetEvent(primitives.aCond[eCond]);
          391  +}
          392  +static void async_sched_yield(void){
          393  +  /* Todo: Find out if win32 offers anything like sched_yield() */
          394  +}
          395  +#else
          396  +
          397  +/* The following block contains the pthreads specific code. */
          398  +#include <pthread.h>
          399  +#include <sched.h>
          400  +
          401  +#define mutex_held(X) pthread_equal(primitives.aHolder[X], pthread_self())
          402  +
          403  +static int  async_os_initialize(void) {return 0;}
          404  +static void async_os_shutdown(void) {}
          405  +
          406  +static struct AsyncPrimitives {
          407  +  pthread_mutex_t aMutex[3];
          408  +  pthread_cond_t aCond[1];
          409  +  pthread_t aHolder[3];
          410  +} primitives = {
          411  +  { PTHREAD_MUTEX_INITIALIZER, 
          412  +    PTHREAD_MUTEX_INITIALIZER, 
          413  +    PTHREAD_MUTEX_INITIALIZER
          414  +  } , {
          415  +    PTHREAD_COND_INITIALIZER
          416  +  } , { 0, 0, 0 }
          417  +};
          418  +
          419  +static void async_mutex_enter(int eMutex){
          420  +  assert( eMutex==0 || eMutex==1 || eMutex==2 );
          421  +  assert( eMutex!=2 || (!mutex_held(0) && !mutex_held(1) && !mutex_held(2)) );
          422  +  assert( eMutex!=1 || (!mutex_held(0) && !mutex_held(1)) );
          423  +  assert( eMutex!=0 || (!mutex_held(0)) );
          424  +  pthread_mutex_lock(&primitives.aMutex[eMutex]);
          425  +  TESTONLY( primitives.aHolder[eMutex] = pthread_self(); )
          426  +}
          427  +static void async_mutex_leave(int eMutex){
          428  +  assert( eMutex==0 || eMutex==1 || eMutex==2 );
          429  +  assert( mutex_held(eMutex) );
          430  +  TESTONLY( primitives.aHolder[eMutex] = 0; )
          431  +  pthread_mutex_unlock(&primitives.aMutex[eMutex]);
          432  +}
          433  +static void async_cond_wait(int eCond, int eMutex){
          434  +  assert( eMutex==0 || eMutex==1 || eMutex==2 );
          435  +  assert( mutex_held(eMutex) );
          436  +  TESTONLY( primitives.aHolder[eMutex] = 0; )
          437  +  pthread_cond_wait(&primitives.aCond[eCond], &primitives.aMutex[eMutex]);
          438  +  TESTONLY( primitives.aHolder[eMutex] = pthread_self(); )
          439  +}
          440  +static void async_cond_signal(int eCond){
          441  +  assert( mutex_held(ASYNC_MUTEX_QUEUE) );
          442  +  pthread_cond_signal(&primitives.aCond[eCond]);
          443  +}
          444  +static void async_sched_yield(void){
          445  +  sched_yield();
          446  +}
          447  +#endif
          448  +/*
          449  +** End of OS specific code.
          450  +*************************************************************************/
          451  +
          452  +#define assert_mutex_is_held(X) assert( mutex_held(X) )
          453  +
          454  +
          455  +#ifndef SQLITE_ASYNC_TWO_FILEHANDLES
          456  +/* #define SQLITE_ASYNC_TWO_FILEHANDLES 0 */
          457  +#define SQLITE_ASYNC_TWO_FILEHANDLES 1
          458  +#endif
          459  +
          460  +/*
          461  +** State information is held in the static variable "async" defined
          462  +** as the following structure.
          463  +**
          464  +** Both async.ioError and async.nFile are protected by async.queueMutex.
          465  +*/
          466  +static struct TestAsyncStaticData {
          467  +  AsyncWrite *pQueueFirst;     /* Next write operation to be processed */
          468  +  AsyncWrite *pQueueLast;      /* Last write operation on the list */
          469  +  AsyncLock *pLock;            /* Linked list of all AsyncLock structures */
          470  +  volatile int ioDelay;        /* Extra delay between write operations */
          471  +  volatile int eHalt;          /* One of the SQLITEASYNC_HALT_XXX values */
          472  +  int ioError;                 /* True if an IO error has occurred */
          473  +  int nFile;                   /* Number of open files (from sqlite pov) */
          474  +} async = { 0,0,0,0,0,0,0 };
          475  +
          476  +/* Possible values of AsyncWrite.op */
          477  +#define ASYNC_NOOP          0
          478  +#define ASYNC_WRITE         1
          479  +#define ASYNC_SYNC          2
          480  +#define ASYNC_TRUNCATE      3
          481  +#define ASYNC_CLOSE         4
          482  +#define ASYNC_DELETE        5
          483  +#define ASYNC_OPENEXCLUSIVE 6
          484  +#define ASYNC_UNLOCK        7
          485  +
          486  +/* Names of opcodes.  Used for debugging only.
          487  +** Make sure these stay in sync with the macros above!
          488  +*/
          489  +static const char *azOpcodeName[] = {
          490  +  "NOOP", "WRITE", "SYNC", "TRUNCATE", "CLOSE", "DELETE", "OPENEX", "UNLOCK"
          491  +};
          492  +
          493  +/*
          494  +** Entries on the write-op queue are instances of the AsyncWrite
          495  +** structure, defined here.
          496  +**
          497  +** The interpretation of the iOffset and nByte variables varies depending 
          498  +** on the value of AsyncWrite.op:
          499  +**
          500  +** ASYNC_NOOP:
          501  +**     No values used.
          502  +**
          503  +** ASYNC_WRITE:
          504  +**     iOffset -> Offset in file to write to.
          505  +**     nByte   -> Number of bytes of data to write (pointed to by zBuf).
          506  +**
          507  +** ASYNC_SYNC:
          508  +**     nByte   -> flags to pass to sqlite3OsSync().
          509  +**
          510  +** ASYNC_TRUNCATE:
          511  +**     iOffset -> Size to truncate file to.
          512  +**     nByte   -> Unused.
          513  +**
          514  +** ASYNC_CLOSE:
          515  +**     iOffset -> Unused.
          516  +**     nByte   -> Unused.
          517  +**
          518  +** ASYNC_DELETE:
          519  +**     iOffset -> Contains the "syncDir" flag.
          520  +**     nByte   -> Number of bytes of zBuf points to (file name).
          521  +**
          522  +** ASYNC_OPENEXCLUSIVE:
          523  +**     iOffset -> Value of "delflag".
          524  +**     nByte   -> Number of bytes of zBuf points to (file name).
          525  +**
          526  +** ASYNC_UNLOCK:
          527  +**     nByte   -> Argument to sqlite3OsUnlock().
          528  +**
          529  +**
          530  +** For an ASYNC_WRITE operation, zBuf points to the data to write to the file. 
          531  +** This space is sqlite3_malloc()d along with the AsyncWrite structure in a
          532  +** single blob, so is deleted when sqlite3_free() is called on the parent 
          533  +** structure.
          534  +*/
          535  +struct AsyncWrite {
          536  +  AsyncFileData *pFileData;    /* File to write data to or sync */
          537  +  int op;                      /* One of ASYNC_xxx etc. */
          538  +  sqlite_int64 iOffset;        /* See above */
          539  +  int nByte;          /* See above */
          540  +  char *zBuf;         /* Data to write to file (or NULL if op!=ASYNC_WRITE) */
          541  +  AsyncWrite *pNext;  /* Next write operation (to any file) */
          542  +};
          543  +
          544  +/*
          545  +** An instance of this structure is created for each distinct open file 
          546  +** (i.e. if two handles are opened on the one file, only one of these
          547  +** structures is allocated) and stored in the async.aLock hash table. The
          548  +** keys for async.aLock are the full pathnames of the opened files.
          549  +**
          550  +** AsyncLock.pList points to the head of a linked list of AsyncFileLock
          551  +** structures, one for each handle currently open on the file.
          552  +**
          553  +** If the opened file is not a main-database (the SQLITE_OPEN_MAIN_DB is
          554  +** not passed to the sqlite3OsOpen() call), or if ENABLE_FILE_LOCKING is 
          555  +** not defined at compile time, variables AsyncLock.pFile and 
          556  +** AsyncLock.eLock are never used. Otherwise, pFile is a file handle
          557  +** opened on the file in question and used to obtain the file-system 
          558  +** locks required by database connections within this process.
          559  +**
          560  +** See comments above the asyncLock() function for more details on 
          561  +** the implementation of database locking used by this backend.
          562  +*/
          563  +struct AsyncLock {
          564  +  char *zFile;
          565  +  int nFile;
          566  +  sqlite3_file *pFile;
          567  +  int eLock;
          568  +  AsyncFileLock *pList;
          569  +  AsyncLock *pNext;           /* Next in linked list headed by async.pLock */
          570  +};
          571  +
          572  +/*
          573  +** An instance of the following structure is allocated along with each
          574  +** AsyncFileData structure (see AsyncFileData.lock), but is only used if the
          575  +** file was opened with the SQLITE_OPEN_MAIN_DB.
          576  +*/
          577  +struct AsyncFileLock {
          578  +  int eLock;                /* Internally visible lock state (sqlite pov) */
          579  +  int eAsyncLock;           /* Lock-state with write-queue unlock */
          580  +  AsyncFileLock *pNext;
          581  +};
          582  +
          583  +/* 
          584  +** The AsyncFile structure is a subclass of sqlite3_file used for 
          585  +** asynchronous IO. 
          586  +**
          587  +** All of the actual data for the structure is stored in the structure
          588  +** pointed to by AsyncFile.pData, which is allocated as part of the
          589  +** sqlite3OsOpen() using sqlite3_malloc(). The reason for this is that the
          590  +** lifetime of the AsyncFile structure is ended by the caller after OsClose()
          591  +** is called, but the data in AsyncFileData may be required by the
          592  +** writer thread after that point.
          593  +*/
          594  +struct AsyncFile {
          595  +  sqlite3_io_methods *pMethod;
          596  +  AsyncFileData *pData;
          597  +};
          598  +struct AsyncFileData {
          599  +  char *zName;               /* Underlying OS filename - used for debugging */
          600  +  int nName;                 /* Number of characters in zName */
          601  +  sqlite3_file *pBaseRead;   /* Read handle to the underlying Os file */
          602  +  sqlite3_file *pBaseWrite;  /* Write handle to the underlying Os file */
          603  +  AsyncFileLock lock;        /* Lock state for this handle */
          604  +  AsyncLock *pLock;          /* AsyncLock object for this file system entry */
          605  +  AsyncWrite closeOp;        /* Preallocated close operation */
          606  +};
          607  +
          608  +/*
          609  +** Add an entry to the end of the global write-op list. pWrite should point 
          610  +** to an AsyncWrite structure allocated using sqlite3_malloc().  The writer
          611  +** thread will call sqlite3_free() to free the structure after the specified
          612  +** operation has been completed.
          613  +**
          614  +** Once an AsyncWrite structure has been added to the list, it becomes the
          615  +** property of the writer thread and must not be read or modified by the
          616  +** caller.  
          617  +*/
          618  +static void addAsyncWrite(AsyncWrite *pWrite){
          619  +  /* We must hold the queue mutex in order to modify the queue pointers */
          620  +  if( pWrite->op!=ASYNC_UNLOCK ){
          621  +    async_mutex_enter(ASYNC_MUTEX_QUEUE);
          622  +  }
          623  +
          624  +  /* Add the record to the end of the write-op queue */
          625  +  assert( !pWrite->pNext );
          626  +  if( async.pQueueLast ){
          627  +    assert( async.pQueueFirst );
          628  +    async.pQueueLast->pNext = pWrite;
          629  +  }else{
          630  +    async.pQueueFirst = pWrite;
          631  +  }
          632  +  async.pQueueLast = pWrite;
          633  +  ASYNC_TRACE(("PUSH %p (%s %s %d)\n", pWrite, azOpcodeName[pWrite->op],
          634  +         pWrite->pFileData ? pWrite->pFileData->zName : "-", pWrite->iOffset));
          635  +
          636  +  if( pWrite->op==ASYNC_CLOSE ){
          637  +    async.nFile--;
          638  +  }
          639  +
          640  +  /* The writer thread might have been idle because there was nothing
          641  +  ** on the write-op queue for it to do.  So wake it up. */
          642  +  async_cond_signal(ASYNC_COND_QUEUE);
          643  +
          644  +  /* Drop the queue mutex */
          645  +  if( pWrite->op!=ASYNC_UNLOCK ){
          646  +    async_mutex_leave(ASYNC_MUTEX_QUEUE);
          647  +  }
          648  +}
          649  +
          650  +/*
          651  +** Increment async.nFile in a thread-safe manner.
          652  +*/
          653  +static void incrOpenFileCount(void){
          654  +  /* We must hold the queue mutex in order to modify async.nFile */
          655  +  async_mutex_enter(ASYNC_MUTEX_QUEUE);
          656  +  if( async.nFile==0 ){
          657  +    async.ioError = SQLITE_OK;
          658  +  }
          659  +  async.nFile++;
          660  +  async_mutex_leave(ASYNC_MUTEX_QUEUE);
          661  +}
          662  +
          663  +/*
          664  +** This is a utility function to allocate and populate a new AsyncWrite
          665  +** structure and insert it (via addAsyncWrite() ) into the global list.
          666  +*/
          667  +static int addNewAsyncWrite(
          668  +  AsyncFileData *pFileData, 
          669  +  int op, 
          670  +  sqlite3_int64 iOffset, 
          671  +  int nByte,
          672  +  const char *zByte
          673  +){
          674  +  AsyncWrite *p;
          675  +  if( op!=ASYNC_CLOSE && async.ioError ){
          676  +    return async.ioError;
          677  +  }
          678  +  p = sqlite3_malloc(sizeof(AsyncWrite) + (zByte?nByte:0));
          679  +  if( !p ){
          680  +    /* The upper layer does not expect operations like OsWrite() to
          681  +    ** return SQLITE_NOMEM. This is partly because under normal conditions
          682  +    ** SQLite is required to do rollback without calling malloc(). So
          683  +    ** if malloc() fails here, treat it as an I/O error. The above
          684  +    ** layer knows how to handle that.
          685  +    */
          686  +    return SQLITE_IOERR;
          687  +  }
          688  +  p->op = op;
          689  +  p->iOffset = iOffset;
          690  +  p->nByte = nByte;
          691  +  p->pFileData = pFileData;
          692  +  p->pNext = 0;
          693  +  if( zByte ){
          694  +    p->zBuf = (char *)&p[1];
          695  +    memcpy(p->zBuf, zByte, nByte);
          696  +  }else{
          697  +    p->zBuf = 0;
          698  +  }
          699  +  addAsyncWrite(p);
          700  +  return SQLITE_OK;
          701  +}
          702  +
          703  +/*
          704  +** Close the file. This just adds an entry to the write-op list, the file is
          705  +** not actually closed.
          706  +*/
          707  +static int asyncClose(sqlite3_file *pFile){
          708  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          709  +
          710  +  /* Unlock the file, if it is locked */
          711  +  async_mutex_enter(ASYNC_MUTEX_LOCK);
          712  +  p->lock.eLock = 0;
          713  +  async_mutex_leave(ASYNC_MUTEX_LOCK);
          714  +
          715  +  addAsyncWrite(&p->closeOp);
          716  +  return SQLITE_OK;
          717  +}
          718  +
          719  +/*
          720  +** Implementation of sqlite3OsWrite() for asynchronous files. Instead of 
          721  +** writing to the underlying file, this function adds an entry to the end of
          722  +** the global AsyncWrite list. Either SQLITE_OK or SQLITE_NOMEM may be
          723  +** returned.
          724  +*/
          725  +static int asyncWrite(
          726  +  sqlite3_file *pFile, 
          727  +  const void *pBuf, 
          728  +  int amt, 
          729  +  sqlite3_int64 iOff
          730  +){
          731  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          732  +  return addNewAsyncWrite(p, ASYNC_WRITE, iOff, amt, pBuf);
          733  +}
          734  +
          735  +/*
          736  +** Read data from the file. First we read from the filesystem, then adjust 
          737  +** the contents of the buffer based on ASYNC_WRITE operations in the 
          738  +** write-op queue.
          739  +**
          740  +** This method holds the mutex from start to finish.
          741  +*/
          742  +static int asyncRead(
          743  +  sqlite3_file *pFile, 
          744  +  void *zOut, 
          745  +  int iAmt, 
          746  +  sqlite3_int64 iOffset
          747  +){
          748  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          749  +  int rc = SQLITE_OK;
          750  +  sqlite3_int64 filesize;
          751  +  int nRead;
          752  +  sqlite3_file *pBase = p->pBaseRead;
          753  +
          754  +  /* Grab the write queue mutex for the duration of the call */
          755  +  async_mutex_enter(ASYNC_MUTEX_QUEUE);
          756  +
          757  +  /* If an I/O error has previously occurred in this virtual file 
          758  +  ** system, then all subsequent operations fail.
          759  +  */
          760  +  if( async.ioError!=SQLITE_OK ){
          761  +    rc = async.ioError;
          762  +    goto asyncread_out;
          763  +  }
          764  +
          765  +  if( pBase->pMethods ){
          766  +    rc = pBase->pMethods->xFileSize(pBase, &filesize);
          767  +    if( rc!=SQLITE_OK ){
          768  +      goto asyncread_out;
          769  +    }
          770  +    nRead = MIN(filesize - iOffset, iAmt);
          771  +    if( nRead>0 ){
          772  +      rc = pBase->pMethods->xRead(pBase, zOut, nRead, iOffset);
          773  +      ASYNC_TRACE(("READ %s %d bytes at %d\n", p->zName, nRead, iOffset));
          774  +    }
          775  +  }
          776  +
          777  +  if( rc==SQLITE_OK ){
          778  +    AsyncWrite *pWrite;
          779  +    char *zName = p->zName;
          780  +
          781  +    for(pWrite=async.pQueueFirst; pWrite; pWrite = pWrite->pNext){
          782  +      if( pWrite->op==ASYNC_WRITE && (
          783  +        (pWrite->pFileData==p) ||
          784  +        (zName && pWrite->pFileData->zName==zName)
          785  +      )){
          786  +        int iBeginOut = (pWrite->iOffset-iOffset);
          787  +        int iBeginIn = -iBeginOut;
          788  +        int nCopy;
          789  +
          790  +        if( iBeginIn<0 ) iBeginIn = 0;
          791  +        if( iBeginOut<0 ) iBeginOut = 0;
          792  +        nCopy = MIN(pWrite->nByte-iBeginIn, iAmt-iBeginOut);
          793  +
          794  +        if( nCopy>0 ){
          795  +          memcpy(&((char *)zOut)[iBeginOut], &pWrite->zBuf[iBeginIn], nCopy);
          796  +          ASYNC_TRACE(("OVERREAD %d bytes at %d\n", nCopy, iBeginOut+iOffset));
          797  +        }
          798  +      }
          799  +    }
          800  +  }
          801  +
          802  +asyncread_out:
          803  +  async_mutex_leave(ASYNC_MUTEX_QUEUE);
          804  +  return rc;
          805  +}
          806  +
          807  +/*
          808  +** Truncate the file to nByte bytes in length. This just adds an entry to 
          809  +** the write-op list, no IO actually takes place.
          810  +*/
          811  +static int asyncTruncate(sqlite3_file *pFile, sqlite3_int64 nByte){
          812  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          813  +  return addNewAsyncWrite(p, ASYNC_TRUNCATE, nByte, 0, 0);
          814  +}
          815  +
          816  +/*
          817  +** Sync the file. This just adds an entry to the write-op list, the 
          818  +** sync() is done later by sqlite3_async_flush().
          819  +*/
          820  +static int asyncSync(sqlite3_file *pFile, int flags){
          821  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          822  +  return addNewAsyncWrite(p, ASYNC_SYNC, 0, flags, 0);
          823  +}
          824  +
          825  +/*
          826  +** Read the size of the file. First we read the size of the file system 
          827  +** entry, then adjust for any ASYNC_WRITE or ASYNC_TRUNCATE operations 
          828  +** currently in the write-op list. 
          829  +**
          830  +** This method holds the mutex from start to finish.
          831  +*/
          832  +int asyncFileSize(sqlite3_file *pFile, sqlite3_int64 *piSize){
          833  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          834  +  int rc = SQLITE_OK;
          835  +  sqlite3_int64 s = 0;
          836  +  sqlite3_file *pBase;
          837  +
          838  +  async_mutex_enter(ASYNC_MUTEX_QUEUE);
          839  +
          840  +  /* Read the filesystem size from the base file. If pBaseRead is NULL, this
          841  +  ** means the file hasn't been opened yet. In this case all relevant data 
          842  +  ** must be in the write-op queue anyway, so we can omit reading from the
          843  +  ** file-system.
          844  +  */
          845  +  pBase = p->pBaseRead;
          846  +  if( pBase->pMethods ){
          847  +    rc = pBase->pMethods->xFileSize(pBase, &s);
          848  +  }
          849  +
          850  +  if( rc==SQLITE_OK ){
          851  +    AsyncWrite *pWrite;
          852  +    for(pWrite=async.pQueueFirst; pWrite; pWrite = pWrite->pNext){
          853  +      if( pWrite->op==ASYNC_DELETE 
          854  +       && p->zName 
          855  +       && strcmp(p->zName, pWrite->zBuf)==0 
          856  +      ){
          857  +        s = 0;
          858  +      }else if( pWrite->pFileData && (
          859  +          (pWrite->pFileData==p) 
          860  +       || (p->zName && pWrite->pFileData->zName==p->zName) 
          861  +      )){
          862  +        switch( pWrite->op ){
          863  +          case ASYNC_WRITE:
          864  +            s = MAX(pWrite->iOffset + (sqlite3_int64)(pWrite->nByte), s);
          865  +            break;
          866  +          case ASYNC_TRUNCATE:
          867  +            s = MIN(s, pWrite->iOffset);
          868  +            break;
          869  +        }
          870  +      }
          871  +    }
          872  +    *piSize = s;
          873  +  }
          874  +  async_mutex_leave(ASYNC_MUTEX_QUEUE);
          875  +  return rc;
          876  +}
          877  +
          878  +/*
          879  +** Lock or unlock the actual file-system entry.
          880  +*/
          881  +static int getFileLock(AsyncLock *pLock){
          882  +  int rc = SQLITE_OK;
          883  +  AsyncFileLock *pIter;
          884  +  int eRequired = 0;
          885  +
          886  +  if( pLock->pFile ){
          887  +    for(pIter=pLock->pList; pIter; pIter=pIter->pNext){
          888  +      assert(pIter->eAsyncLock>=pIter->eLock);
          889  +      if( pIter->eAsyncLock>eRequired ){
          890  +        eRequired = pIter->eAsyncLock;
          891  +        assert(eRequired>=0 && eRequired<=SQLITE_LOCK_EXCLUSIVE);
          892  +      }
          893  +    }
          894  +
          895  +    if( eRequired>pLock->eLock ){
          896  +      rc = pLock->pFile->pMethods->xLock(pLock->pFile, eRequired);
          897  +      if( rc==SQLITE_OK ){
          898  +        pLock->eLock = eRequired;
          899  +      }
          900  +    }
          901  +    else if( eRequired<pLock->eLock && eRequired<=SQLITE_LOCK_SHARED ){
          902  +      rc = pLock->pFile->pMethods->xUnlock(pLock->pFile, eRequired);
          903  +      if( rc==SQLITE_OK ){
          904  +        pLock->eLock = eRequired;
          905  +      }
          906  +    }
          907  +  }
          908  +
          909  +  return rc;
          910  +}
          911  +
          912  +/*
          913  +** Return the AsyncLock structure from the global async.pLock list 
          914  +** associated with the file-system entry identified by path zName 
          915  +** (a string of nName bytes). If no such structure exists, return 0.
          916  +*/
          917  +static AsyncLock *findLock(const char *zName, int nName){
          918  +  AsyncLock *p = async.pLock;
          919  +  while( p && (p->nFile!=nName || memcmp(p->zFile, zName, nName)) ){
          920  +    p = p->pNext;
          921  +  }
          922  +  return p;
          923  +}
          924  +
          925  +/*
          926  +** The following two methods - asyncLock() and asyncUnlock() - are used
          927  +** to obtain and release locks on database files opened with the
          928  +** asynchronous backend.
          929  +*/
          930  +static int asyncLock(sqlite3_file *pFile, int eLock){
          931  +  int rc = SQLITE_OK;
          932  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          933  +
          934  +  if( p->zName ){
          935  +    async_mutex_enter(ASYNC_MUTEX_LOCK);
          936  +    if( p->lock.eLock<eLock ){
          937  +      AsyncLock *pLock = p->pLock;
          938  +      AsyncFileLock *pIter;
          939  +      assert(pLock && pLock->pList);
          940  +      for(pIter=pLock->pList; pIter; pIter=pIter->pNext){
          941  +        if( pIter!=&p->lock && (
          942  +          (eLock==SQLITE_LOCK_EXCLUSIVE && pIter->eLock>=SQLITE_LOCK_SHARED) ||
          943  +          (eLock==SQLITE_LOCK_PENDING && pIter->eLock>=SQLITE_LOCK_RESERVED) ||
          944  +          (eLock==SQLITE_LOCK_RESERVED && pIter->eLock>=SQLITE_LOCK_RESERVED) ||
          945  +          (eLock==SQLITE_LOCK_SHARED && pIter->eLock>=SQLITE_LOCK_PENDING)
          946  +        )){
          947  +          rc = SQLITE_BUSY;
          948  +        }
          949  +      }
          950  +      if( rc==SQLITE_OK ){
          951  +        p->lock.eLock = eLock;
          952  +        p->lock.eAsyncLock = MAX(p->lock.eAsyncLock, eLock);
          953  +      }
          954  +      assert(p->lock.eAsyncLock>=p->lock.eLock);
          955  +      if( rc==SQLITE_OK ){
          956  +        rc = getFileLock(pLock);
          957  +      }
          958  +    }
          959  +    async_mutex_leave(ASYNC_MUTEX_LOCK);
          960  +  }
          961  +
          962  +  ASYNC_TRACE(("LOCK %d (%s) rc=%d\n", eLock, p->zName, rc));
          963  +  return rc;
          964  +}
          965  +static int asyncUnlock(sqlite3_file *pFile, int eLock){
          966  +  int rc = SQLITE_OK;
          967  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          968  +  if( p->zName ){
          969  +    AsyncFileLock *pLock = &p->lock;
          970  +    async_mutex_enter(ASYNC_MUTEX_QUEUE);
          971  +    async_mutex_enter(ASYNC_MUTEX_LOCK);
          972  +    pLock->eLock = MIN(pLock->eLock, eLock);
          973  +    rc = addNewAsyncWrite(p, ASYNC_UNLOCK, 0, eLock, 0);
          974  +    async_mutex_leave(ASYNC_MUTEX_LOCK);
          975  +    async_mutex_leave(ASYNC_MUTEX_QUEUE);
          976  +  }
          977  +  return rc;
          978  +}
          979  +
          980  +/*
          981  +** This function is called when the pager layer first opens a database file
          982  +** and is checking for a hot-journal.
          983  +*/
          984  +static int asyncCheckReservedLock(sqlite3_file *pFile, int *pResOut){
          985  +  int ret = 0;
          986  +  AsyncFileLock *pIter;
          987  +  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
          988  +
          989  +  async_mutex_enter(ASYNC_MUTEX_LOCK);
          990  +  for(pIter=p->pLock->pList; pIter; pIter=pIter->pNext){
          991  +    if( pIter->eLock>=SQLITE_LOCK_RESERVED ){
          992  +      ret = 1;
          993  +    }
          994  +  }
          995  +  async_mutex_leave(ASYNC_MUTEX_LOCK);
          996  +
          997  +  ASYNC_TRACE(("CHECK-LOCK %d (%s)\n", ret, p->zName));
          998  +  *pResOut = ret;
          999  +  return SQLITE_OK;
         1000  +}
         1001  +
         1002  +/* 
         1003  +** sqlite3_file_control() implementation.
         1004  +*/
         1005  +static int asyncFileControl(sqlite3_file *id, int op, void *pArg){
         1006  +  switch( op ){
         1007  +    case SQLITE_FCNTL_LOCKSTATE: {
         1008  +      async_mutex_enter(ASYNC_MUTEX_LOCK);
         1009  +      *(int*)pArg = ((AsyncFile*)id)->pData->lock.eLock;
         1010  +      async_mutex_leave(ASYNC_MUTEX_LOCK);
         1011  +      return SQLITE_OK;
         1012  +    }
         1013  +  }
         1014  +  return SQLITE_ERROR;
         1015  +}
         1016  +
         1017  +/* 
         1018  +** Return the device characteristics and sector-size of the device. It
         1019  +** is not tricky to implement these correctly, as this backend might 
         1020  +** not have an open file handle at this point.
         1021  +*/
         1022  +static int asyncSectorSize(sqlite3_file *pFile){
         1023  +  return 512;
         1024  +}
         1025  +static int asyncDeviceCharacteristics(sqlite3_file *pFile){
         1026  +  return 0;
         1027  +}
         1028  +
         1029  +static int unlinkAsyncFile(AsyncFileData *pData){
         1030  +  AsyncFileLock **ppIter;
         1031  +  int rc = SQLITE_OK;
         1032  +
         1033  +  if( pData->zName ){
         1034  +    AsyncLock *pLock = pData->pLock;
         1035  +    for(ppIter=&pLock->pList; *ppIter; ppIter=&((*ppIter)->pNext)){
         1036  +      if( (*ppIter)==&pData->lock ){
         1037  +        *ppIter = pData->lock.pNext;
         1038  +        break;
         1039  +      }
         1040  +    }
         1041  +    if( !pLock->pList ){
         1042  +      AsyncLock **pp;
         1043  +      if( pLock->pFile ){
         1044  +        pLock->pFile->pMethods->xClose(pLock->pFile);
         1045  +      }
         1046  +      for(pp=&async.pLock; *pp!=pLock; pp=&((*pp)->pNext));
         1047  +      *pp = pLock->pNext;
         1048  +      sqlite3_free(pLock);
         1049  +    }else{
         1050  +      rc = getFileLock(pLock);
         1051  +    }
         1052  +  }
         1053  +
         1054  +  return rc;
         1055  +}
         1056  +
         1057  +/*
         1058  +** The parameter passed to this function is a copy of a 'flags' parameter
         1059  +** passed to this modules xOpen() method. This function returns true
         1060  +** if the file should be opened asynchronously, or false if it should
         1061  +** be opened immediately.
         1062  +**
         1063  +** If the file is to be opened asynchronously, then asyncOpen() will add
         1064  +** an entry to the event queue and the file will not actually be opened
         1065  +** until the event is processed. Otherwise, the file is opened directly
         1066  +** by the caller.
         1067  +*/
         1068  +static int doAsynchronousOpen(int flags){
         1069  +  return (flags&SQLITE_OPEN_CREATE) && (
         1070  +      (flags&SQLITE_OPEN_MAIN_JOURNAL) ||
         1071  +      (flags&SQLITE_OPEN_TEMP_JOURNAL) ||
         1072  +      (flags&SQLITE_OPEN_DELETEONCLOSE)
         1073  +  );
         1074  +}
         1075  +
         1076  +/*
         1077  +** Open a file.
         1078  +*/
         1079  +static int asyncOpen(
         1080  +  sqlite3_vfs *pAsyncVfs,
         1081  +  const char *zName,
         1082  +  sqlite3_file *pFile,
         1083  +  int flags,
         1084  +  int *pOutFlags
         1085  +){
         1086  +  static sqlite3_io_methods async_methods = {
         1087  +    1,                               /* iVersion */
         1088  +    asyncClose,                      /* xClose */
         1089  +    asyncRead,                       /* xRead */
         1090  +    asyncWrite,                      /* xWrite */
         1091  +    asyncTruncate,                   /* xTruncate */
         1092  +    asyncSync,                       /* xSync */
         1093  +    asyncFileSize,                   /* xFileSize */
         1094  +    asyncLock,                       /* xLock */
         1095  +    asyncUnlock,                     /* xUnlock */
         1096  +    asyncCheckReservedLock,          /* xCheckReservedLock */
         1097  +    asyncFileControl,                /* xFileControl */
         1098  +    asyncSectorSize,                 /* xSectorSize */
         1099  +    asyncDeviceCharacteristics       /* xDeviceCharacteristics */
         1100  +  };
         1101  +
         1102  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1103  +  AsyncFile *p = (AsyncFile *)pFile;
         1104  +  int nName = 0;
         1105  +  int rc = SQLITE_OK;
         1106  +  int nByte;
         1107  +  AsyncFileData *pData;
         1108  +  AsyncLock *pLock = 0;
         1109  +  char *z;
         1110  +  int isAsyncOpen = doAsynchronousOpen(flags);
         1111  +
         1112  +  /* If zName is NULL, then the upper layer is requesting an anonymous file */
         1113  +  if( zName ){
         1114  +    nName = strlen(zName)+1;
         1115  +  }
         1116  +
         1117  +  nByte = (
         1118  +    sizeof(AsyncFileData) +        /* AsyncFileData structure */
         1119  +    2 * pVfs->szOsFile +           /* AsyncFileData.pBaseRead and pBaseWrite */
         1120  +    nName                          /* AsyncFileData.zName */
         1121  +  ); 
         1122  +  z = sqlite3_malloc(nByte);
         1123  +  if( !z ){
         1124  +    return SQLITE_NOMEM;
         1125  +  }
         1126  +  memset(z, 0, nByte);
         1127  +  pData = (AsyncFileData*)z;
         1128  +  z += sizeof(pData[0]);
         1129  +  pData->pBaseRead = (sqlite3_file*)z;
         1130  +  z += pVfs->szOsFile;
         1131  +  pData->pBaseWrite = (sqlite3_file*)z;
         1132  +  pData->closeOp.pFileData = pData;
         1133  +  pData->closeOp.op = ASYNC_CLOSE;
         1134  +
         1135  +  if( zName ){
         1136  +    z += pVfs->szOsFile;
         1137  +    pData->zName = z;
         1138  +    pData->nName = nName;
         1139  +    memcpy(pData->zName, zName, nName);
         1140  +  }
         1141  +
         1142  +  if( !isAsyncOpen ){
         1143  +    int flagsout;
         1144  +    rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseRead, flags, &flagsout);
         1145  +    if( rc==SQLITE_OK && (flagsout&SQLITE_OPEN_READWRITE) ){
         1146  +      rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseWrite, flags, 0);
         1147  +    }
         1148  +    if( pOutFlags ){
         1149  +      *pOutFlags = flagsout;
         1150  +    }
         1151  +  }
         1152  +
         1153  +  async_mutex_enter(ASYNC_MUTEX_LOCK);
         1154  +
         1155  +  if( zName && rc==SQLITE_OK ){
         1156  +    pLock = findLock(pData->zName, pData->nName);
         1157  +    if( !pLock ){
         1158  +      int nByte = pVfs->szOsFile + sizeof(AsyncLock) + pData->nName + 1; 
         1159  +      pLock = (AsyncLock *)sqlite3_malloc(nByte);
         1160  +      if( pLock ){
         1161  +        memset(pLock, 0, nByte);
         1162  +#ifdef ENABLE_FILE_LOCKING
         1163  +        if( flags&SQLITE_OPEN_MAIN_DB ){
         1164  +          pLock->pFile = (sqlite3_file *)&pLock[1];
         1165  +          rc = pVfs->xOpen(pVfs, pData->zName, pLock->pFile, flags, 0);
         1166  +          if( rc!=SQLITE_OK ){
         1167  +            sqlite3_free(pLock);
         1168  +            pLock = 0;
         1169  +          }
         1170  +        }
         1171  +#endif
         1172  +        if( pLock ){
         1173  +          pLock->nFile = pData->nName;
         1174  +          pLock->zFile = &((char *)(&pLock[1]))[pVfs->szOsFile];
         1175  +          memcpy(pLock->zFile, pData->zName, pLock->nFile);
         1176  +          pLock->pNext = async.pLock;
         1177  +          async.pLock = pLock;
         1178  +        }
         1179  +      }else{
         1180  +        rc = SQLITE_NOMEM;
         1181  +      }
         1182  +    }
         1183  +  }
         1184  +
         1185  +  if( rc==SQLITE_OK ){
         1186  +    p->pMethod = &async_methods;
         1187  +    p->pData = pData;
         1188  +
         1189  +    /* Link AsyncFileData.lock into the linked list of 
         1190  +    ** AsyncFileLock structures for this file.
         1191  +    */
         1192  +    if( zName ){
         1193  +      pData->lock.pNext = pLock->pList;
         1194  +      pLock->pList = &pData->lock;
         1195  +      pData->zName = pLock->zFile;
         1196  +    }
         1197  +  }else{
         1198  +    if( pData->pBaseRead->pMethods ){
         1199  +      pData->pBaseRead->pMethods->xClose(pData->pBaseRead);
         1200  +    }
         1201  +    if( pData->pBaseWrite->pMethods ){
         1202  +      pData->pBaseWrite->pMethods->xClose(pData->pBaseWrite);
         1203  +    }
         1204  +    sqlite3_free(pData);
         1205  +  }
         1206  +
         1207  +  async_mutex_leave(ASYNC_MUTEX_LOCK);
         1208  +
         1209  +  if( rc==SQLITE_OK ){
         1210  +    incrOpenFileCount();
         1211  +    pData->pLock = pLock;
         1212  +  }
         1213  +
         1214  +  if( rc==SQLITE_OK && isAsyncOpen ){
         1215  +    rc = addNewAsyncWrite(pData, ASYNC_OPENEXCLUSIVE, (sqlite3_int64)flags,0,0);
         1216  +    if( rc==SQLITE_OK ){
         1217  +      if( pOutFlags ) *pOutFlags = flags;
         1218  +    }else{
         1219  +      async_mutex_enter(ASYNC_MUTEX_LOCK);
         1220  +      unlinkAsyncFile(pData);
         1221  +      async_mutex_leave(ASYNC_MUTEX_LOCK);
         1222  +      sqlite3_free(pData);
         1223  +    }
         1224  +  }
         1225  +  if( rc!=SQLITE_OK ){
         1226  +    p->pMethod = 0;
         1227  +  }
         1228  +  return rc;
         1229  +}
         1230  +
         1231  +/*
         1232  +** Implementation of sqlite3OsDelete. Add an entry to the end of the 
         1233  +** write-op queue to perform the delete.
         1234  +*/
         1235  +static int asyncDelete(sqlite3_vfs *pAsyncVfs, const char *z, int syncDir){
         1236  +  return addNewAsyncWrite(0, ASYNC_DELETE, syncDir, strlen(z)+1, z);
         1237  +}
         1238  +
         1239  +/*
         1240  +** Implementation of sqlite3OsAccess. This method holds the mutex from
         1241  +** start to finish.
         1242  +*/
         1243  +static int asyncAccess(
         1244  +  sqlite3_vfs *pAsyncVfs, 
         1245  +  const char *zName, 
         1246  +  int flags,
         1247  +  int *pResOut
         1248  +){
         1249  +  int rc;
         1250  +  int ret;
         1251  +  AsyncWrite *p;
         1252  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1253  +
         1254  +  assert(flags==SQLITE_ACCESS_READWRITE 
         1255  +      || flags==SQLITE_ACCESS_READ 
         1256  +      || flags==SQLITE_ACCESS_EXISTS 
         1257  +  );
         1258  +
         1259  +  async_mutex_enter(ASYNC_MUTEX_QUEUE);
         1260  +  rc = pVfs->xAccess(pVfs, zName, flags, &ret);
         1261  +  if( rc==SQLITE_OK && flags==SQLITE_ACCESS_EXISTS ){
         1262  +    for(p=async.pQueueFirst; p; p = p->pNext){
         1263  +      if( p->op==ASYNC_DELETE && 0==strcmp(p->zBuf, zName) ){
         1264  +        ret = 0;
         1265  +      }else if( p->op==ASYNC_OPENEXCLUSIVE 
         1266  +             && p->pFileData->zName
         1267  +             && 0==strcmp(p->pFileData->zName, zName) 
         1268  +      ){
         1269  +        ret = 1;
         1270  +      }
         1271  +    }
         1272  +  }
         1273  +  ASYNC_TRACE(("ACCESS(%s): %s = %d\n", 
         1274  +    flags==SQLITE_ACCESS_READWRITE?"read-write":
         1275  +    flags==SQLITE_ACCESS_READ?"read":"exists"
         1276  +    , zName, ret)
         1277  +  );
         1278  +  async_mutex_leave(ASYNC_MUTEX_QUEUE);
         1279  +  *pResOut = ret;
         1280  +  return rc;
         1281  +}
         1282  +
         1283  +/*
         1284  +** Fill in zPathOut with the full path to the file identified by zPath.
         1285  +*/
         1286  +static int asyncFullPathname(
         1287  +  sqlite3_vfs *pAsyncVfs, 
         1288  +  const char *zPath, 
         1289  +  int nPathOut,
         1290  +  char *zPathOut
         1291  +){
         1292  +  int rc;
         1293  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1294  +  rc = pVfs->xFullPathname(pVfs, zPath, nPathOut, zPathOut);
         1295  +
         1296  +  /* Because of the way intra-process file locking works, this backend
         1297  +  ** needs to return a canonical path. The following block assumes the
         1298  +  ** file-system uses unix style paths. 
         1299  +  */
         1300  +  if( rc==SQLITE_OK ){
         1301  +    int i, j;
         1302  +    int n = nPathOut;
         1303  +    char *z = zPathOut;
         1304  +    while( n>1 && z[n-1]=='/' ){ n--; }
         1305  +    for(i=j=0; i<n; i++){
         1306  +      if( z[i]=='/' ){
         1307  +        if( z[i+1]=='/' ) continue;
         1308  +        if( z[i+1]=='.' && i+2<n && z[i+2]=='/' ){
         1309  +          i += 1;
         1310  +          continue;
         1311  +        }
         1312  +        if( z[i+1]=='.' && i+3<n && z[i+2]=='.' && z[i+3]=='/' ){
         1313  +          while( j>0 && z[j-1]!='/' ){ j--; }
         1314  +          if( j>0 ){ j--; }
         1315  +          i += 2;
         1316  +          continue;
         1317  +        }
         1318  +      }
         1319  +      z[j++] = z[i];
         1320  +    }
         1321  +    z[j] = 0;
         1322  +  }
         1323  +
         1324  +  return rc;
         1325  +}
         1326  +static void *asyncDlOpen(sqlite3_vfs *pAsyncVfs, const char *zPath){
         1327  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1328  +  return pVfs->xDlOpen(pVfs, zPath);
         1329  +}
         1330  +static void asyncDlError(sqlite3_vfs *pAsyncVfs, int nByte, char *zErrMsg){
         1331  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1332  +  pVfs->xDlError(pVfs, nByte, zErrMsg);
         1333  +}
         1334  +static void (*asyncDlSym(
         1335  +  sqlite3_vfs *pAsyncVfs, 
         1336  +  void *pHandle, 
         1337  +  const char *zSymbol
         1338  +))(void){
         1339  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1340  +  return pVfs->xDlSym(pVfs, pHandle, zSymbol);
         1341  +}
         1342  +static void asyncDlClose(sqlite3_vfs *pAsyncVfs, void *pHandle){
         1343  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1344  +  pVfs->xDlClose(pVfs, pHandle);
         1345  +}
         1346  +static int asyncRandomness(sqlite3_vfs *pAsyncVfs, int nByte, char *zBufOut){
         1347  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1348  +  return pVfs->xRandomness(pVfs, nByte, zBufOut);
         1349  +}
         1350  +static int asyncSleep(sqlite3_vfs *pAsyncVfs, int nMicro){
         1351  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1352  +  return pVfs->xSleep(pVfs, nMicro);
         1353  +}
         1354  +static int asyncCurrentTime(sqlite3_vfs *pAsyncVfs, double *pTimeOut){
         1355  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
         1356  +  return pVfs->xCurrentTime(pVfs, pTimeOut);
         1357  +}
         1358  +
         1359  +static sqlite3_vfs async_vfs = {
         1360  +  1,                    /* iVersion */
         1361  +  sizeof(AsyncFile),    /* szOsFile */
         1362  +  0,                    /* mxPathname */
         1363  +  0,                    /* pNext */
         1364  +  SQLITEASYNC_VFSNAME,  /* zName */
         1365  +  0,                    /* pAppData */
         1366  +  asyncOpen,            /* xOpen */
         1367  +  asyncDelete,          /* xDelete */
         1368  +  asyncAccess,          /* xAccess */
         1369  +  asyncFullPathname,    /* xFullPathname */
         1370  +  asyncDlOpen,          /* xDlOpen */
         1371  +  asyncDlError,         /* xDlError */
         1372  +  asyncDlSym,           /* xDlSym */
         1373  +  asyncDlClose,         /* xDlClose */
         1374  +  asyncRandomness,      /* xDlError */
         1375  +  asyncSleep,           /* xDlSym */
         1376  +  asyncCurrentTime      /* xDlClose */
         1377  +};
         1378  +
         1379  +/* 
         1380  +** This procedure runs in a separate thread, reading messages off of the
         1381  +** write queue and processing them one by one.  
         1382  +**
         1383  +** If async.writerHaltNow is true, then this procedure exits
         1384  +** after processing a single message.
         1385  +**
         1386  +** If async.writerHaltWhenIdle is true, then this procedure exits when
         1387  +** the write queue is empty.
         1388  +**
         1389  +** If both of the above variables are false, this procedure runs
         1390  +** indefinately, waiting for operations to be added to the write queue
         1391  +** and processing them in the order in which they arrive.
         1392  +**
         1393  +** An artifical delay of async.ioDelay milliseconds is inserted before
         1394  +** each write operation in order to simulate the effect of a slow disk.
         1395  +**
         1396  +** Only one instance of this procedure may be running at a time.
         1397  +*/
         1398  +static void asyncWriterThread(void){
         1399  +  sqlite3_vfs *pVfs = (sqlite3_vfs *)(async_vfs.pAppData);
         1400  +  AsyncWrite *p = 0;
         1401  +  int rc = SQLITE_OK;
         1402  +  int holdingMutex = 0;
         1403  +
         1404  +  async_mutex_enter(ASYNC_MUTEX_WRITER);
         1405  +
         1406  +  while( async.eHalt!=SQLITEASYNC_HALT_NOW ){
         1407  +    int doNotFree = 0;
         1408  +    sqlite3_file *pBase = 0;
         1409  +
         1410  +    if( !holdingMutex ){
         1411  +      async_mutex_enter(ASYNC_MUTEX_QUEUE);
         1412  +    }
         1413  +    while( (p = async.pQueueFirst)==0 ){
         1414  +      if( async.eHalt!=SQLITEASYNC_HALT_NEVER ){
         1415  +        async_mutex_leave(ASYNC_MUTEX_QUEUE);
         1416  +        break;
         1417  +      }else{
         1418  +        ASYNC_TRACE(("IDLE\n"));
         1419  +        async_cond_wait(ASYNC_COND_QUEUE, ASYNC_MUTEX_QUEUE);
         1420  +        ASYNC_TRACE(("WAKEUP\n"));
         1421  +      }
         1422  +    }
         1423  +    if( p==0 ) break;
         1424  +    holdingMutex = 1;
         1425  +
         1426  +    /* Right now this thread is holding the mutex on the write-op queue.
         1427  +    ** Variable 'p' points to the first entry in the write-op queue. In
         1428  +    ** the general case, we hold on to the mutex for the entire body of
         1429  +    ** the loop. 
         1430  +    **
         1431  +    ** However in the cases enumerated below, we relinquish the mutex,
         1432  +    ** perform the IO, and then re-request the mutex before removing 'p' from
         1433  +    ** the head of the write-op queue. The idea is to increase concurrency with
         1434  +    ** sqlite threads.
         1435  +    **
         1436  +    **     * An ASYNC_CLOSE operation.
         1437  +    **     * An ASYNC_OPENEXCLUSIVE operation. For this one, we relinquish 
         1438  +    **       the mutex, call the underlying xOpenExclusive() function, then
         1439  +    **       re-aquire the mutex before seting the AsyncFile.pBaseRead 
         1440  +    **       variable.
         1441  +    **     * ASYNC_SYNC and ASYNC_WRITE operations, if 
         1442  +    **       SQLITE_ASYNC_TWO_FILEHANDLES was set at compile time and two
         1443  +    **       file-handles are open for the particular file being "synced".
         1444  +    */
         1445  +    if( async.ioError!=SQLITE_OK && p->op!=ASYNC_CLOSE ){
         1446  +      p->op = ASYNC_NOOP;
         1447  +    }
         1448  +    if( p->pFileData ){
         1449  +      pBase = p->pFileData->pBaseWrite;
         1450  +      if( 
         1451  +        p->op==ASYNC_CLOSE || 
         1452  +        p->op==ASYNC_OPENEXCLUSIVE ||
         1453  +        (pBase->pMethods && (p->op==ASYNC_SYNC || p->op==ASYNC_WRITE) ) 
         1454  +      ){
         1455  +        async_mutex_leave(ASYNC_MUTEX_QUEUE);
         1456  +        holdingMutex = 0;
         1457  +      }
         1458  +      if( !pBase->pMethods ){
         1459  +        pBase = p->pFileData->pBaseRead;
         1460  +      }
         1461  +    }
         1462  +
         1463  +    switch( p->op ){
         1464  +      case ASYNC_NOOP:
         1465  +        break;
         1466  +
         1467  +      case ASYNC_WRITE:
         1468  +        assert( pBase );
         1469  +        ASYNC_TRACE(("WRITE %s %d bytes at %d\n",
         1470  +                p->pFileData->zName, p->nByte, p->iOffset));
         1471  +        rc = pBase->pMethods->xWrite(pBase, (void *)(p->zBuf), p->nByte, p->iOffset);
         1472  +        break;
         1473  +
         1474  +      case ASYNC_SYNC:
         1475  +        assert( pBase );
         1476  +        ASYNC_TRACE(("SYNC %s\n", p->pFileData->zName));
         1477  +        rc = pBase->pMethods->xSync(pBase, p->nByte);
         1478  +        break;
         1479  +
         1480  +      case ASYNC_TRUNCATE:
         1481  +        assert( pBase );
         1482  +        ASYNC_TRACE(("TRUNCATE %s to %d bytes\n", 
         1483  +                p->pFileData->zName, p->iOffset));
         1484  +        rc = pBase->pMethods->xTruncate(pBase, p->iOffset);
         1485  +        break;
         1486  +
         1487  +      case ASYNC_CLOSE: {
         1488  +        AsyncFileData *pData = p->pFileData;
         1489  +        ASYNC_TRACE(("CLOSE %s\n", p->pFileData->zName));
         1490  +        if( pData->pBaseWrite->pMethods ){
         1491  +          pData->pBaseWrite->pMethods->xClose(pData->pBaseWrite);
         1492  +        }
         1493  +        if( pData->pBaseRead->pMethods ){
         1494  +          pData->pBaseRead->pMethods->xClose(pData->pBaseRead);
         1495  +        }
         1496  +
         1497  +        /* Unlink AsyncFileData.lock from the linked list of AsyncFileLock 
         1498  +        ** structures for this file. Obtain the async.lockMutex mutex 
         1499  +        ** before doing so.
         1500  +        */
         1501  +        async_mutex_enter(ASYNC_MUTEX_LOCK);
         1502  +        rc = unlinkAsyncFile(pData);
         1503  +        async_mutex_leave(ASYNC_MUTEX_LOCK);
         1504  +
         1505  +        if( !holdingMutex ){
         1506  +          async_mutex_enter(ASYNC_MUTEX_QUEUE);
         1507  +          holdingMutex = 1;
         1508  +        }
         1509  +        assert_mutex_is_held(ASYNC_MUTEX_QUEUE);
         1510  +        async.pQueueFirst = p->pNext;
         1511  +        sqlite3_free(pData);
         1512  +        doNotFree = 1;
         1513  +        break;
         1514  +      }
         1515  +
         1516  +      case ASYNC_UNLOCK: {
         1517  +        AsyncWrite *pIter;
         1518  +        AsyncFileData *pData = p->pFileData;
         1519  +        int eLock = p->nByte;
         1520  +
         1521  +        /* When a file is locked by SQLite using the async backend, it is 
         1522  +        ** locked within the 'real' file-system synchronously. When it is
         1523  +        ** unlocked, an ASYNC_UNLOCK event is added to the write-queue to
         1524  +        ** unlock the file asynchronously. The design of the async backend
         1525  +        ** requires that the 'real' file-system file be locked from the
         1526  +        ** time that SQLite first locks it (and probably reads from it)
         1527  +        ** until all asynchronous write events that were scheduled before
         1528  +        ** SQLite unlocked the file have been processed.
         1529  +        **
         1530  +        ** This is more complex if SQLite locks and unlocks the file multiple
         1531  +        ** times in quick succession. For example, if SQLite does: 
         1532  +        ** 
         1533  +        **   lock, write, unlock, lock, write, unlock
         1534  +        **
         1535  +        ** Each "lock" operation locks the file immediately. Each "write" 
         1536  +        ** and "unlock" operation adds an event to the event queue. If the
         1537  +        ** second "lock" operation is performed before the first "unlock"
         1538  +        ** operation has been processed asynchronously, then the first
         1539  +        ** "unlock" cannot be safely processed as is, since this would mean
         1540  +        ** the file was unlocked when the second "write" operation is
         1541  +        ** processed. To work around this, when processing an ASYNC_UNLOCK
         1542  +        ** operation, SQLite:
         1543  +        **
         1544  +        **   1) Unlocks the file to the minimum of the argument passed to
         1545  +        **      the xUnlock() call and the current lock from SQLite's point
         1546  +        **      of view, and
         1547  +        **
         1548  +        **   2) Only unlocks the file at all if this event is the last
         1549  +        **      ASYNC_UNLOCK event on this file in the write-queue.
         1550  +        */ 
         1551  +        assert( holdingMutex==1 );
         1552  +        assert( async.pQueueFirst==p );
         1553  +        for(pIter=async.pQueueFirst->pNext; pIter; pIter=pIter->pNext){
         1554  +          if( pIter->pFileData==pData && pIter->op==ASYNC_UNLOCK ) break;
         1555  +        }
         1556  +        if( !pIter ){
         1557  +          async_mutex_enter(ASYNC_MUTEX_LOCK);
         1558  +          pData->lock.eAsyncLock = MIN(
         1559  +              pData->lock.eAsyncLock, MAX(pData->lock.eLock, eLock)
         1560  +          );
         1561  +          assert(pData->lock.eAsyncLock>=pData->lock.eLock);
         1562  +          rc = getFileLock(pData->pLock);
         1563  +          async_mutex_leave(ASYNC_MUTEX_LOCK);
         1564  +        }
         1565  +        break;
         1566  +      }
         1567  +
         1568  +      case ASYNC_DELETE:
         1569  +        ASYNC_TRACE(("DELETE %s\n", p->zBuf));
         1570  +        rc = pVfs->xDelete(pVfs, p->zBuf, (int)p->iOffset);
         1571  +        break;
         1572  +
         1573  +      case ASYNC_OPENEXCLUSIVE: {
         1574  +        int flags = (int)p->iOffset;
         1575  +        AsyncFileData *pData = p->pFileData;
         1576  +        ASYNC_TRACE(("OPEN %s flags=%d\n", p->zBuf, (int)p->iOffset));
         1577  +        assert(pData->pBaseRead->pMethods==0 && pData->pBaseWrite->pMethods==0);
         1578  +        rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseRead, flags, 0);
         1579  +        assert( holdingMutex==0 );
         1580  +        async_mutex_enter(ASYNC_MUTEX_QUEUE);
         1581  +        holdingMutex = 1;
         1582  +        break;
         1583  +      }
         1584  +
         1585  +      default: assert(!"Illegal value for AsyncWrite.op");
         1586  +    }
         1587  +
         1588  +    /* If we didn't hang on to the mutex during the IO op, obtain it now
         1589  +    ** so that the AsyncWrite structure can be safely removed from the 
         1590  +    ** global write-op queue.
         1591  +    */
         1592  +    if( !holdingMutex ){
         1593  +      async_mutex_enter(ASYNC_MUTEX_QUEUE);
         1594  +      holdingMutex = 1;
         1595  +    }
         1596  +    /* ASYNC_TRACE(("UNLINK %p\n", p)); */
         1597  +    if( p==async.pQueueLast ){
         1598  +      async.pQueueLast = 0;
         1599  +    }
         1600  +    if( !doNotFree ){
         1601  +      assert_mutex_is_held(ASYNC_MUTEX_QUEUE);
         1602  +      async.pQueueFirst = p->pNext;
         1603  +      sqlite3_free(p);
         1604  +    }
         1605  +    assert( holdingMutex );
         1606  +
         1607  +    /* An IO error has occurred. We cannot report the error back to the
         1608  +    ** connection that requested the I/O since the error happened 
         1609  +    ** asynchronously.  The connection has already moved on.  There 
         1610  +    ** really is nobody to report the error to.
         1611  +    **
         1612  +    ** The file for which the error occurred may have been a database or
         1613  +    ** journal file. Regardless, none of the currently queued operations
         1614  +    ** associated with the same database should now be performed. Nor should
         1615  +    ** any subsequently requested IO on either a database or journal file 
         1616  +    ** handle for the same database be accepted until the main database
         1617  +    ** file handle has been closed and reopened.
         1618  +    **
         1619  +    ** Furthermore, no further IO should be queued or performed on any file
         1620  +    ** handle associated with a database that may have been part of a 
         1621  +    ** multi-file transaction that included the database associated with 
         1622  +    ** the IO error (i.e. a database ATTACHed to the same handle at some 
         1623  +    ** point in time).
         1624  +    */
         1625  +    if( rc!=SQLITE_OK ){
         1626  +      async.ioError = rc;
         1627  +    }
         1628  +
         1629  +    if( async.ioError && !async.pQueueFirst ){
         1630  +      async_mutex_enter(ASYNC_MUTEX_LOCK);
         1631  +      if( 0==async.pLock ){
         1632  +        async.ioError = SQLITE_OK;
         1633  +      }
         1634  +      async_mutex_leave(ASYNC_MUTEX_LOCK);
         1635  +    }
         1636  +
         1637  +    /* Drop the queue mutex before continuing to the next write operation
         1638  +    ** in order to give other threads a chance to work with the write queue.
         1639  +    */
         1640  +    if( !async.pQueueFirst || !async.ioError ){
         1641  +      async_mutex_leave(ASYNC_MUTEX_QUEUE);
         1642  +      holdingMutex = 0;
         1643  +      if( async.ioDelay>0 ){
         1644  +        pVfs->xSleep(pVfs, async.ioDelay);
         1645  +      }else{
         1646  +        async_sched_yield();
         1647  +      }
         1648  +    }
         1649  +  }
         1650  +  
         1651  +  async_mutex_leave(ASYNC_MUTEX_WRITER);
         1652  +  return;
         1653  +}
         1654  +
         1655  +/*
         1656  +** Install the asynchronous VFS.
         1657  +*/ 
         1658  +int sqlite3async_initialize(const char *zParent, int isDefault){
         1659  +  int rc = SQLITE_OK;
         1660  +  if( async_vfs.pAppData==0 ){
         1661  +    sqlite3_vfs *pParent = sqlite3_vfs_find(zParent);
         1662  +    if( !pParent || async_os_initialize() ){
         1663  +      rc = SQLITE_ERROR;
         1664  +    }else if( SQLITE_OK!=(rc = sqlite3_vfs_register(&async_vfs, isDefault)) ){
         1665  +      async_os_shutdown();
         1666  +    }else{
         1667  +      async_vfs.pAppData = (void *)pParent;
         1668  +      async_vfs.mxPathname = ((sqlite3_vfs *)async_vfs.pAppData)->mxPathname;
         1669  +    }
         1670  +  }
         1671  +  return rc;
         1672  +}
         1673  +
         1674  +/*
         1675  +** Uninstall the asynchronous VFS.
         1676  +*/
         1677  +void sqlite3async_shutdown(void){
         1678  +  if( async_vfs.pAppData ){
         1679  +    async_os_shutdown();
         1680  +    sqlite3_vfs_unregister((sqlite3_vfs *)&async_vfs);
         1681  +    async_vfs.pAppData = 0;
         1682  +  }
         1683  +}
         1684  +
         1685  +/*
         1686  +** Process events on the write-queue.
         1687  +*/
         1688  +void sqlite3async_run(void){
         1689  +  asyncWriterThread();
         1690  +}
         1691  +
         1692  +/*
         1693  +** Control/configure the asynchronous IO system.
         1694  +*/
         1695  +int sqlite3async_control(int op, ...){
         1696  +  va_list ap;
         1697  +  va_start(ap, op);
         1698  +  switch( op ){
         1699  +    case SQLITEASYNC_HALT: {
         1700  +      int eWhen = va_arg(ap, int);
         1701  +      if( eWhen!=SQLITEASYNC_HALT_NEVER
         1702  +       && eWhen!=SQLITEASYNC_HALT_NOW
         1703  +       && eWhen!=SQLITEASYNC_HALT_IDLE
         1704  +      ){
         1705  +        return SQLITE_ERROR;
         1706  +      }
         1707  +      async.eHalt = eWhen;
         1708  +      async_mutex_enter(ASYNC_MUTEX_QUEUE);
         1709  +      async_cond_signal(ASYNC_COND_QUEUE);
         1710  +      async_mutex_leave(ASYNC_MUTEX_QUEUE);
         1711  +      break;
         1712  +    }
         1713  +
         1714  +    case SQLITEASYNC_DELAY: {
         1715  +      int iDelay = va_arg(ap, int);
         1716  +      async.ioDelay = iDelay;
         1717  +      break;
         1718  +    }
         1719  +      
         1720  +    case SQLITEASYNC_GET_HALT: {
         1721  +      int *peWhen = va_arg(ap, int *);
         1722  +      *peWhen = async.eHalt;
         1723  +      break;
         1724  +    }
         1725  +    case SQLITEASYNC_GET_DELAY: {
         1726  +      int *piDelay = va_arg(ap, int *);
         1727  +      *piDelay = async.ioDelay;
         1728  +      break;
         1729  +    }
         1730  +
         1731  +    default:
         1732  +      return SQLITE_ERROR;
         1733  +  }
         1734  +  return SQLITE_OK;
         1735  +}
         1736  +
         1737  +#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ASYNCIO) */
         1738  +

Added ext/async/sqlite3async.h.

            1  +
            2  +#ifndef __SQLITEASYNC_H_
            3  +#define __SQLITEASYNC_H_ 1
            4  +
            5  +#define SQLITEASYNC_VFSNAME "sqlite3async"
            6  +
            7  +/*
            8  +** Install the asynchronous IO VFS.
            9  +*/ 
           10  +int sqlite3async_initialize(const char *zParent, int isDefault);
           11  +
           12  +/*
           13  +** Uninstall the asynchronous IO VFS.
           14  +*/ 
           15  +void sqlite3async_shutdown();
           16  +
           17  +/*
           18  +** Process events on the write-queue.
           19  +*/
           20  +void sqlite3async_run();
           21  +
           22  +/*
           23  +** Control/configure the asynchronous IO system.
           24  +*/
           25  +int sqlite3async_control(int op, ...);
           26  +
           27  +/*
           28  +** Values that can be used as the first argument to sqlite3async_control().
           29  +*/
           30  +#define SQLITEASYNC_HALT       1
           31  +#define SQLITEASYNC_DELAY      2
           32  +#define SQLITEASYNC_GET_HALT   3
           33  +#define SQLITEASYNC_GET_DELAY  4
           34  +
           35  +/*
           36  +** If the first argument to sqlite3async_control() is SQLITEASYNC_HALT,
           37  +** the second argument should be one of the following.
           38  +*/
           39  +#define SQLITEASYNC_HALT_NEVER 0       /* Never halt (default value) */
           40  +#define SQLITEASYNC_HALT_NOW   1       /* Halt as soon as possible */
           41  +#define SQLITEASYNC_HALT_IDLE  2       /* Halt when write-queue is empty */
           42  +
           43  +#endif        /* ifndef __SQLITEASYNC_H_ */
           44  +

Changes to main.mk.

    42     42   # build the SQLite library and testing tools.
    43     43   ################################################################################
    44     44   
    45     45   # This is how we compile
    46     46   #
    47     47   TCCX =  $(TCC) $(OPTS) -I. -I$(TOP)/src -I$(TOP) 
    48     48   TCCX += -I$(TOP)/ext/rtree -I$(TOP)/ext/icu -I$(TOP)/ext/fts3
           49  +TCCX += -I$(TOP)/ext/async
    49     50   
    50     51   # Object files for the SQLite library.
    51     52   #
    52     53   LIBOBJ+= alter.o analyze.o attach.o auth.o \
    53     54            backup.o bitvec.o btmutex.o btree.o build.o \
    54     55            callback.o complete.o date.o delete.o expr.o fault.o \
    55     56            fts3.o fts3_expr.o fts3_hash.o fts3_icu.o fts3_porter.o \
................................................................................
   251    252     $(TOP)/src/os_os2.c $(TOP)/src/os_unix.c $(TOP)/src/os_win.c                 \
   252    253     $(TOP)/src/pager.c $(TOP)/src/pragma.c $(TOP)/src/prepare.c                  \
   253    254     $(TOP)/src/printf.c $(TOP)/src/random.c $(TOP)/src/pcache.c                  \
   254    255     $(TOP)/src/pcache1.c $(TOP)/src/select.c $(TOP)/src/tokenize.c               \
   255    256     $(TOP)/src/utf.c $(TOP)/src/util.c $(TOP)/src/vdbeapi.c $(TOP)/src/vdbeaux.c \
   256    257     $(TOP)/src/vdbe.c $(TOP)/src/vdbemem.c $(TOP)/src/where.c parse.c            \
   257    258     $(TOP)/ext/fts3/fts3.c $(TOP)/ext/fts3/fts3_expr.c                           \
   258         -  $(TOP)/ext/fts3/fts3_tokenizer.c 
          259  +  $(TOP)/ext/fts3/fts3_tokenizer.c                                             \
          260  +  $(TOP)/ext/async/sqlite3async.c
   259    261   
   260    262   # Header files used by all library source files.
   261    263   #
   262    264   HDR = \
   263    265      $(TOP)/src/btree.h \
   264    266      $(TOP)/src/btreeInt.h \
   265    267      $(TOP)/src/hash.h \

Changes to src/test_async.c.

     6      6   **
     7      7   **    May you do good and not evil.
     8      8   **    May you find forgiveness for yourself and forgive others.
     9      9   **    May you share freely, never taking more than you give.
    10     10   **
    11     11   *************************************************************************
    12     12   **
    13         -** $Id: test_async.c,v 1.58 2009/04/21 18:20:45 danielk1977 Exp $
           13  +** $Id: test_async.c,v 1.59 2009/04/23 14:58:40 danielk1977 Exp $
    14     14   **
    15         -** This file contains an example implementation of an asynchronous IO 
    16         -** backend for SQLite.
    17         -**
    18         -** WHAT IS ASYNCHRONOUS I/O?
    19         -**
    20         -** With asynchronous I/O, write requests are handled by a separate thread
    21         -** running in the background.  This means that the thread that initiates
    22         -** a database write does not have to wait for (sometimes slow) disk I/O
    23         -** to occur.  The write seems to happen very quickly, though in reality
    24         -** it is happening at its usual slow pace in the background.
    25         -**
    26         -** Asynchronous I/O appears to give better responsiveness, but at a price.
    27         -** You lose the Durable property.  With the default I/O backend of SQLite,
    28         -** once a write completes, you know that the information you wrote is
    29         -** safely on disk.  With the asynchronous I/O, this is not the case.  If
    30         -** your program crashes or if a power loss occurs after the database
    31         -** write but before the asynchronous write thread has completed, then the
    32         -** database change might never make it to disk and the next user of the
    33         -** database might not see your change.
    34         -**
    35         -** You lose Durability with asynchronous I/O, but you still retain the
    36         -** other parts of ACID:  Atomic,  Consistent, and Isolated.  Many
    37         -** appliations get along fine without the Durablity.
    38         -**
    39         -** HOW IT WORKS
    40         -**
    41         -** Asynchronous I/O works by creating a special SQLite "vfs" structure
    42         -** and registering it with sqlite3_vfs_register(). When files opened via 
    43         -** this vfs are written to (using sqlite3OsWrite()), the data is not 
    44         -** written directly to disk, but is placed in the "write-queue" to be
    45         -** handled by the background thread.
    46         -**
    47         -** When files opened with the asynchronous vfs are read from 
    48         -** (using sqlite3OsRead()), the data is read from the file on 
    49         -** disk and the write-queue, so that from the point of view of
    50         -** the vfs reader the OsWrite() appears to have already completed.
    51         -**
    52         -** The special vfs is registered (and unregistered) by calls to 
    53         -** function asyncEnable() (see below).
    54         -**
    55         -** LIMITATIONS
    56         -**
    57         -** This demonstration code is deliberately kept simple in order to keep
    58         -** the main ideas clear and easy to understand.  Real applications that
    59         -** want to do asynchronous I/O might want to add additional capabilities.
    60         -** For example, in this demonstration if writes are happening at a steady
    61         -** stream that exceeds the I/O capability of the background writer thread,
    62         -** the queue of pending write operations will grow without bound until we
    63         -** run out of memory.  Users of this technique may want to keep track of
    64         -** the quantity of pending writes and stop accepting new write requests
    65         -** when the buffer gets to be too big.
    66         -**
    67         -** LOCKING + CONCURRENCY
    68         -**
    69         -** Multiple connections from within a single process that use this
    70         -** implementation of asynchronous IO may access a single database
    71         -** file concurrently. From the point of view of the user, if all
    72         -** connections are from within a single process, there is no difference
    73         -** between the concurrency offered by "normal" SQLite and SQLite
    74         -** using the asynchronous backend.
    75         -**
    76         -** If connections from within multiple processes may access the
    77         -** database file, the ENABLE_FILE_LOCKING symbol (see below) must be
    78         -** defined. If it is not defined, then no locks are established on 
    79         -** the database file. In this case, if multiple processes access 
    80         -** the database file, corruption will quickly result.
    81         -**
    82         -** If ENABLE_FILE_LOCKING is defined (the default), then connections 
    83         -** from within multiple processes may access a single database file 
    84         -** without risking corruption. However concurrency is reduced as
    85         -** follows:
    86         -**
    87         -**   * When a connection using asynchronous IO begins a database
    88         -**     transaction, the database is locked immediately. However the
    89         -**     lock is not released until after all relevant operations
    90         -**     in the write-queue have been flushed to disk. This means
    91         -**     (for example) that the database may remain locked for some 
    92         -**     time after a "COMMIT" or "ROLLBACK" is issued.
    93         -**
    94         -**   * If an application using asynchronous IO executes transactions
    95         -**     in quick succession, other database users may be effectively
    96         -**     locked out of the database. This is because when a BEGIN
    97         -**     is executed, a database lock is established immediately. But
    98         -**     when the corresponding COMMIT or ROLLBACK occurs, the lock
    99         -**     is not released until the relevant part of the write-queue 
   100         -**     has been flushed through. As a result, if a COMMIT is followed
   101         -**     by a BEGIN before the write-queue is flushed through, the database 
   102         -**     is never unlocked,preventing other processes from accessing 
   103         -**     the database.
   104         -**
   105         -** Defining ENABLE_FILE_LOCKING when using an NFS or other remote 
   106         -** file-system may slow things down, as synchronous round-trips to the 
   107         -** server may be required to establish database file locks.
           15  +** This file contains a binding of the asynchronous IO extension interface
           16  +** (defined in ext/async/sqlite3async.h) to Tcl.
   108     17   */
   109         -#define ENABLE_FILE_LOCKING
   110     18   
   111         -#ifndef SQLITE_AMALGAMATION
   112         -# include "sqliteInt.h"
   113         -# include <assert.h>
   114         -# include <string.h>
   115         -#endif
           19  +#define TCL_THREADS 
   116     20   #include <tcl.h>
   117     21   
   118         -/*
   119         -** This test uses pthreads and hence only works on unix and with
   120         -** a threadsafe build of SQLite.
   121         -*/
   122         -#if SQLITE_OS_UNIX && SQLITE_THREADSAFE
   123         -
   124         -/*
   125         -** This demo uses pthreads.  If you do not have a pthreads implementation
   126         -** for your operating system, you will need to recode the threading 
   127         -** logic.
   128         -*/
   129         -#include <pthread.h>
   130         -#include <sched.h>
   131         -
   132         -/* Useful macros used in several places */
   133         -#define MIN(x,y) ((x)<(y)?(x):(y))
   134         -#define MAX(x,y) ((x)>(y)?(x):(y))
   135         -
   136         -/* Forward references */
   137         -typedef struct AsyncWrite AsyncWrite;
   138         -typedef struct AsyncFile AsyncFile;
   139         -typedef struct AsyncFileData AsyncFileData;
   140         -typedef struct AsyncFileLock AsyncFileLock;
   141         -typedef struct AsyncLock AsyncLock;
   142         -
   143         -/* Enable for debugging */
   144         -static int sqlite3async_trace = 0;
   145         -# define ASYNC_TRACE(X) if( sqlite3async_trace ) asyncTrace X
   146         -static void asyncTrace(const char *zFormat, ...){
   147         -  char *z;
   148         -  va_list ap;
   149         -  va_start(ap, zFormat);
   150         -  z = sqlite3_vmprintf(zFormat, ap);
   151         -  va_end(ap);
   152         -  fprintf(stderr, "[%d] %s", (int)pthread_self(), z);
   153         -  sqlite3_free(z);
   154         -}
   155         -
   156         -/*
   157         -** THREAD SAFETY NOTES
   158         -**
   159         -** Basic rules:
   160         -**
   161         -**     * Both read and write access to the global write-op queue must be 
   162         -**       protected by the async.queueMutex. As are the async.ioError and
   163         -**       async.nFile variables.
   164         -**
   165         -**     * The async.pLock list and all AsyncLock and AsyncFileLock
   166         -**       structures must be protected by the async.lockMutex mutex.
   167         -**
   168         -**     * The file handles from the underlying system are not assumed to 
   169         -**       be thread safe.
   170         -**
   171         -**     * See the last two paragraphs under "The Writer Thread" for
   172         -**       an assumption to do with file-handle synchronization by the Os.
   173         -**
   174         -** Deadlock prevention:
   175         -**
   176         -**     There are three mutex used by the system: the "writer" mutex, 
   177         -**     the "queue" mutex and the "lock" mutex. Rules are:
   178         -**
   179         -**     * It is illegal to block on the writer mutex when any other mutex
   180         -**       are held, and 
   181         -**
   182         -**     * It is illegal to block on the queue mutex when the lock mutex
   183         -**       is held.
   184         -**
   185         -**     i.e. mutex's must be grabbed in the order "writer", "queue", "lock".
   186         -**
   187         -** File system operations (invoked by SQLite thread):
   188         -**
   189         -**     xOpen
   190         -**     xDelete
   191         -**     xFileExists
   192         -**
   193         -** File handle operations (invoked by SQLite thread):
   194         -**
   195         -**         asyncWrite, asyncClose, asyncTruncate, asyncSync 
   196         -**    
   197         -**     The operations above add an entry to the global write-op list. They
   198         -**     prepare the entry, acquire the async.queueMutex momentarily while
   199         -**     list pointers are  manipulated to insert the new entry, then release
   200         -**     the mutex and signal the writer thread to wake up in case it happens
   201         -**     to be asleep.
   202         -**
   203         -**    
   204         -**         asyncRead, asyncFileSize.
   205         -**
   206         -**     Read operations. Both of these read from both the underlying file
   207         -**     first then adjust their result based on pending writes in the 
   208         -**     write-op queue.   So async.queueMutex is held for the duration
   209         -**     of these operations to prevent other threads from changing the
   210         -**     queue in mid operation.
   211         -**    
   212         -**
   213         -**         asyncLock, asyncUnlock, asyncCheckReservedLock
   214         -**    
   215         -**     These primitives implement in-process locking using a hash table
   216         -**     on the file name.  Files are locked correctly for connections coming
   217         -**     from the same process.  But other processes cannot see these locks
   218         -**     and will therefore not honor them.
   219         -**
   220         -**
   221         -** The writer thread:
   222         -**
   223         -**     The async.writerMutex is used to make sure only there is only
   224         -**     a single writer thread running at a time.
   225         -**
   226         -**     Inside the writer thread is a loop that works like this:
   227         -**
   228         -**         WHILE (write-op list is not empty)
   229         -**             Do IO operation at head of write-op list
   230         -**             Remove entry from head of write-op list
   231         -**         END WHILE
   232         -**
   233         -**     The async.queueMutex is always held during the <write-op list is 
   234         -**     not empty> test, and when the entry is removed from the head
   235         -**     of the write-op list. Sometimes it is held for the interim
   236         -**     period (while the IO is performed), and sometimes it is
   237         -**     relinquished. It is relinquished if (a) the IO op is an
   238         -**     ASYNC_CLOSE or (b) when the file handle was opened, two of
   239         -**     the underlying systems handles were opened on the same
   240         -**     file-system entry.
   241         -**
   242         -**     If condition (b) above is true, then one file-handle 
   243         -**     (AsyncFile.pBaseRead) is used exclusively by sqlite threads to read the
   244         -**     file, the other (AsyncFile.pBaseWrite) by sqlite3_async_flush() 
   245         -**     threads to perform write() operations. This means that read 
   246         -**     operations are not blocked by asynchronous writes (although 
   247         -**     asynchronous writes may still be blocked by reads).
   248         -**
   249         -**     This assumes that the OS keeps two handles open on the same file
   250         -**     properly in sync. That is, any read operation that starts after a
   251         -**     write operation on the same file system entry has completed returns
   252         -**     data consistent with the write. We also assume that if one thread 
   253         -**     reads a file while another is writing it all bytes other than the
   254         -**     ones actually being written contain valid data.
   255         -**
   256         -**     If the above assumptions are not true, set the preprocessor symbol
   257         -**     SQLITE_ASYNC_TWO_FILEHANDLES to 0.
   258         -*/
   259         -
   260         -#ifndef SQLITE_ASYNC_TWO_FILEHANDLES
   261         -/* #define SQLITE_ASYNC_TWO_FILEHANDLES 0 */
   262         -#define SQLITE_ASYNC_TWO_FILEHANDLES 1
   263         -#endif
   264         -
   265         -/*
   266         -** State information is held in the static variable "async" defined
   267         -** as the following structure.
   268         -**
   269         -** Both async.ioError and async.nFile are protected by async.queueMutex.
   270         -*/
   271         -static struct TestAsyncStaticData {
   272         -  pthread_mutex_t lockMutex;   /* For access to aLock hash table */
   273         -  pthread_mutex_t queueMutex;  /* Mutex for access to write operation queue */
   274         -  pthread_mutex_t writerMutex; /* Prevents multiple writer threads */
   275         -  pthread_cond_t queueSignal;  /* For waking up sleeping writer thread */
   276         -  pthread_cond_t emptySignal;  /* Notify when the write queue is empty */
   277         -  AsyncWrite *pQueueFirst;     /* Next write operation to be processed */
   278         -  AsyncWrite *pQueueLast;      /* Last write operation on the list */
   279         -  AsyncLock *pLock;            /* Linked list of all AsyncLock structures */
   280         -  volatile int ioDelay;             /* Extra delay between write operations */
   281         -  volatile int writerHaltWhenIdle;  /* Writer thread halts when queue empty */
   282         -  volatile int writerHaltNow;       /* Writer thread halts after next op */
   283         -  int ioError;                 /* True if an IO error has occurred */
   284         -  int nFile;                   /* Number of open files (from sqlite pov) */
   285         -} async = {
   286         -  PTHREAD_MUTEX_INITIALIZER,
   287         -  PTHREAD_MUTEX_INITIALIZER,
   288         -  PTHREAD_MUTEX_INITIALIZER,
   289         -  PTHREAD_COND_INITIALIZER,
   290         -  PTHREAD_COND_INITIALIZER,
   291         -};
   292         -
   293         -/* Possible values of AsyncWrite.op */
   294         -#define ASYNC_NOOP          0
   295         -#define ASYNC_WRITE         1
   296         -#define ASYNC_SYNC          2
   297         -#define ASYNC_TRUNCATE      3
   298         -#define ASYNC_CLOSE         4
   299         -#define ASYNC_DELETE        5
   300         -#define ASYNC_OPENEXCLUSIVE 6
   301         -#define ASYNC_UNLOCK        7
   302         -
   303         -/* Names of opcodes.  Used for debugging only.
   304         -** Make sure these stay in sync with the macros above!
   305         -*/
   306         -static const char *azOpcodeName[] = {
   307         -  "NOOP", "WRITE", "SYNC", "TRUNCATE", "CLOSE", "DELETE", "OPENEX", "UNLOCK"
   308         -};
   309         -
   310         -/*
   311         -** Entries on the write-op queue are instances of the AsyncWrite
   312         -** structure, defined here.
   313         -**
   314         -** The interpretation of the iOffset and nByte variables varies depending 
   315         -** on the value of AsyncWrite.op:
   316         -**
   317         -** ASYNC_NOOP:
   318         -**     No values used.
   319         -**
   320         -** ASYNC_WRITE:
   321         -**     iOffset -> Offset in file to write to.
   322         -**     nByte   -> Number of bytes of data to write (pointed to by zBuf).
   323         -**
   324         -** ASYNC_SYNC:
   325         -**     nByte   -> flags to pass to sqlite3OsSync().
   326         -**
   327         -** ASYNC_TRUNCATE:
   328         -**     iOffset -> Size to truncate file to.
   329         -**     nByte   -> Unused.
   330         -**
   331         -** ASYNC_CLOSE:
   332         -**     iOffset -> Unused.
   333         -**     nByte   -> Unused.
   334         -**
   335         -** ASYNC_DELETE:
   336         -**     iOffset -> Contains the "syncDir" flag.
   337         -**     nByte   -> Number of bytes of zBuf points to (file name).
   338         -**
   339         -** ASYNC_OPENEXCLUSIVE:
   340         -**     iOffset -> Value of "delflag".
   341         -**     nByte   -> Number of bytes of zBuf points to (file name).
   342         -**
   343         -** ASYNC_UNLOCK:
   344         -**     nByte   -> Argument to sqlite3OsUnlock().
   345         -**
   346         -**
   347         -** For an ASYNC_WRITE operation, zBuf points to the data to write to the file. 
   348         -** This space is sqlite3_malloc()d along with the AsyncWrite structure in a
   349         -** single blob, so is deleted when sqlite3_free() is called on the parent 
   350         -** structure.
   351         -*/
   352         -struct AsyncWrite {
   353         -  AsyncFileData *pFileData;    /* File to write data to or sync */
   354         -  int op;                      /* One of ASYNC_xxx etc. */
   355         -  sqlite_int64 iOffset;        /* See above */
   356         -  int nByte;          /* See above */
   357         -  char *zBuf;         /* Data to write to file (or NULL if op!=ASYNC_WRITE) */
   358         -  AsyncWrite *pNext;  /* Next write operation (to any file) */
   359         -};
   360         -
   361         -/*
   362         -** An instance of this structure is created for each distinct open file 
   363         -** (i.e. if two handles are opened on the one file, only one of these
   364         -** structures is allocated) and stored in the async.aLock hash table. The
   365         -** keys for async.aLock are the full pathnames of the opened files.
   366         -**
   367         -** AsyncLock.pList points to the head of a linked list of AsyncFileLock
   368         -** structures, one for each handle currently open on the file.
   369         -**
   370         -** If the opened file is not a main-database (the SQLITE_OPEN_MAIN_DB is
   371         -** not passed to the sqlite3OsOpen() call), or if ENABLE_FILE_LOCKING is 
   372         -** not defined at compile time, variables AsyncLock.pFile and 
   373         -** AsyncLock.eLock are never used. Otherwise, pFile is a file handle
   374         -** opened on the file in question and used to obtain the file-system 
   375         -** locks required by database connections within this process.
   376         -**
   377         -** See comments above the asyncLock() function for more details on 
   378         -** the implementation of database locking used by this backend.
   379         -*/
   380         -struct AsyncLock {
   381         -  char *zFile;
   382         -  int nFile;
   383         -  sqlite3_file *pFile;
   384         -  int eLock;
   385         -  AsyncFileLock *pList;
   386         -  AsyncLock *pNext;           /* Next in linked list headed by async.pLock */
   387         -};
   388         -
   389         -/*
   390         -** An instance of the following structure is allocated along with each
   391         -** AsyncFileData structure (see AsyncFileData.lock), but is only used if the
   392         -** file was opened with the SQLITE_OPEN_MAIN_DB.
   393         -*/
   394         -struct AsyncFileLock {
   395         -  int eLock;                /* Internally visible lock state (sqlite pov) */
   396         -  int eAsyncLock;           /* Lock-state with write-queue unlock */
   397         -  AsyncFileLock *pNext;
   398         -};
   399         -
   400         -/* 
   401         -** The AsyncFile structure is a subclass of sqlite3_file used for 
   402         -** asynchronous IO. 
   403         -**
   404         -** All of the actual data for the structure is stored in the structure
   405         -** pointed to by AsyncFile.pData, which is allocated as part of the
   406         -** sqlite3OsOpen() using sqlite3_malloc(). The reason for this is that the
   407         -** lifetime of the AsyncFile structure is ended by the caller after OsClose()
   408         -** is called, but the data in AsyncFileData may be required by the
   409         -** writer thread after that point.
   410         -*/
   411         -struct AsyncFile {
   412         -  sqlite3_io_methods *pMethod;
   413         -  AsyncFileData *pData;
   414         -};
   415         -struct AsyncFileData {
   416         -  char *zName;               /* Underlying OS filename - used for debugging */
   417         -  int nName;                 /* Number of characters in zName */
   418         -  sqlite3_file *pBaseRead;   /* Read handle to the underlying Os file */
   419         -  sqlite3_file *pBaseWrite;  /* Write handle to the underlying Os file */
   420         -  AsyncFileLock lock;        /* Lock state for this handle */
   421         -  AsyncLock *pLock;          /* AsyncLock object for this file system entry */
   422         -  AsyncWrite closeOp;        /* Preallocated close operation */
   423         -};
   424         -
   425         -/*
   426         -** The following async_XXX functions are debugging wrappers around the
   427         -** corresponding pthread_XXX functions:
   428         -**
   429         -**     pthread_mutex_lock();
   430         -**     pthread_mutex_unlock();
   431         -**     pthread_mutex_trylock();
   432         -**     pthread_cond_wait();
   433         -**
   434         -** It is illegal to pass any mutex other than those stored in the
   435         -** following global variables of these functions.
   436         -**
   437         -**     async.queueMutex
   438         -**     async.writerMutex
   439         -**     async.lockMutex
   440         -**
   441         -** If NDEBUG is defined, these wrappers do nothing except call the 
   442         -** corresponding pthreads function. If NDEBUG is not defined, then the
   443         -** following variables are used to store the thread-id (as returned
   444         -** by pthread_self()) currently holding the mutex, or 0 otherwise:
   445         -**
   446         -**     asyncdebug.queueMutexHolder
   447         -**     asyncdebug.writerMutexHolder
   448         -**     asyncdebug.lockMutexHolder
   449         -**
   450         -** These variables are used by some assert() statements that verify
   451         -** the statements made in the "Deadlock Prevention" notes earlier
   452         -** in this file.
   453         -*/
   454         -#ifndef NDEBUG
   455         -
   456         -static struct TestAsyncDebugData {
   457         -  pthread_t lockMutexHolder;
   458         -  pthread_t queueMutexHolder;
   459         -  pthread_t writerMutexHolder;
   460         -} asyncdebug = {0, 0, 0};
   461         -
   462         -/*
   463         -** Wrapper around pthread_mutex_lock(). Checks that we have not violated
   464         -** the anti-deadlock rules (see "Deadlock prevention" above).
   465         -*/
   466         -static int async_mutex_lock(pthread_mutex_t *pMutex){
   467         -  int iIdx;
   468         -  int rc;
   469         -  pthread_mutex_t *aMutex = (pthread_mutex_t *)(&async);
   470         -  pthread_t *aHolder = (pthread_t *)(&asyncdebug);
   471         -
   472         -  /* The code in this 'ifndef NDEBUG' block depends on a certain alignment
   473         -   * of the variables in TestAsyncStaticData and TestAsyncDebugData. The
   474         -   * following assert() statements check that this has not been changed.
   475         -   *
   476         -   * Really, these only need to be run once at startup time.
   477         -   */
   478         -  assert(&(aMutex[0])==&async.lockMutex);
   479         -  assert(&(aMutex[1])==&async.queueMutex);
   480         -  assert(&(aMutex[2])==&async.writerMutex);
   481         -  assert(&(aHolder[0])==&asyncdebug.lockMutexHolder);
   482         -  assert(&(aHolder[1])==&asyncdebug.queueMutexHolder);
   483         -  assert(&(aHolder[2])==&asyncdebug.writerMutexHolder);
   484         -
   485         -  assert( pthread_self()!=0 );
   486         -  for(iIdx=0; iIdx<3; iIdx++){
   487         -    if( pMutex==&aMutex[iIdx] ) break;
   488         -
   489         -    /* This is the key assert(). Here we are checking that if the caller
   490         -     * is trying to block on async.writerMutex, neither of the other two
   491         -     * mutex are held. If the caller is trying to block on async.queueMutex,
   492         -     * lockMutex is not held.
   493         -     */
   494         -    assert(!pthread_equal(aHolder[iIdx], pthread_self()));
   495         -  }
   496         -  assert(iIdx<3);
   497         -
   498         -  rc = pthread_mutex_lock(pMutex);
   499         -  if( rc==0 ){
   500         -    assert(aHolder[iIdx]==0);
   501         -    aHolder[iIdx] = pthread_self();
   502         -  }
   503         -  return rc;
   504         -}
   505         -
   506         -/*
   507         -** Wrapper around pthread_mutex_unlock().
   508         -*/
   509         -static int async_mutex_unlock(pthread_mutex_t *pMutex){
   510         -  int iIdx;
   511         -  int rc;
   512         -  pthread_mutex_t *aMutex = (pthread_mutex_t *)(&async);
   513         -  pthread_t *aHolder = (pthread_t *)(&asyncdebug);
   514         -
   515         -  for(iIdx=0; iIdx<3; iIdx++){
   516         -    if( pMutex==&aMutex[iIdx] ) break;
   517         -  }
   518         -  assert(iIdx<3);
   519         -
   520         -  assert(pthread_equal(aHolder[iIdx], pthread_self()));
   521         -  aHolder[iIdx] = 0;
   522         -  rc = pthread_mutex_unlock(pMutex);
   523         -  assert(rc==0);
   524         -
   525         -  return 0;
   526         -}
   527         -
   528         -/*
   529         -** Wrapper around pthread_mutex_trylock().
   530         -*/
   531         -static int async_mutex_trylock(pthread_mutex_t *pMutex){
   532         -  int iIdx;
   533         -  int rc;
   534         -  pthread_mutex_t *aMutex = (pthread_mutex_t *)(&async);
   535         -  pthread_t *aHolder = (pthread_t *)(&asyncdebug);
   536         -
   537         -  for(iIdx=0; iIdx<3; iIdx++){
   538         -    if( pMutex==&aMutex[iIdx] ) break;
   539         -  }
   540         -  assert(iIdx<3);
   541         -
   542         -  rc = pthread_mutex_trylock(pMutex);
   543         -  if( rc==0 ){
   544         -    assert(aHolder[iIdx]==0);
   545         -    aHolder[iIdx] = pthread_self();
   546         -  }
   547         -  return rc;
   548         -}
   549         -
   550         -/*
   551         -** Wrapper around pthread_cond_wait().
   552         -*/
   553         -static int async_cond_wait(pthread_cond_t *pCond, pthread_mutex_t *pMutex){
   554         -  int iIdx;
   555         -  int rc;
   556         -  pthread_mutex_t *aMutex = (pthread_mutex_t *)(&async);
   557         -  pthread_t *aHolder = (pthread_t *)(&asyncdebug);
   558         -
   559         -  for(iIdx=0; iIdx<3; iIdx++){
   560         -    if( pMutex==&aMutex[iIdx] ) break;
   561         -  }
   562         -  assert(iIdx<3);
   563         -
   564         -  assert(pthread_equal(aHolder[iIdx],pthread_self()));
   565         -  aHolder[iIdx] = 0;
   566         -  rc = pthread_cond_wait(pCond, pMutex);
   567         -  if( rc==0 ){
   568         -    aHolder[iIdx] = pthread_self();
   569         -  }
   570         -  return rc;
   571         -}
   572         -
   573         -/*
   574         -** Assert that the mutex is held by the current thread.
   575         -*/
   576         -static void assert_mutex_is_held(pthread_mutex_t *pMutex){
   577         -  int iIdx;
   578         -  pthread_mutex_t *aMutex = (pthread_mutex_t *)(&async);
   579         -  pthread_t *aHolder = (pthread_t *)(&asyncdebug);
   580         -
   581         -  for(iIdx=0; iIdx<3; iIdx++){
   582         -    if( pMutex==&aMutex[iIdx] ) break;
   583         -  }
   584         -  assert(iIdx<3);
   585         -  assert( aHolder[iIdx]==pthread_self() );
   586         -}
   587         -
   588         -/* Call our async_XX wrappers instead of selected pthread_XX functions */
   589         -#define pthread_mutex_lock    async_mutex_lock
   590         -#define pthread_mutex_unlock  async_mutex_unlock
   591         -#define pthread_mutex_trylock async_mutex_trylock
   592         -#define pthread_cond_wait     async_cond_wait
   593         -
   594         -#else    /* if defined(NDEBUG) */
   595         -
   596         -#define assert_mutex_is_held(X)    /* A no-op when not debugging */
   597         -
   598         -#endif   /* !defined(NDEBUG) */
   599         -
   600         -/*
   601         -** Add an entry to the end of the global write-op list. pWrite should point 
   602         -** to an AsyncWrite structure allocated using sqlite3_malloc().  The writer
   603         -** thread will call sqlite3_free() to free the structure after the specified
   604         -** operation has been completed.
   605         -**
   606         -** Once an AsyncWrite structure has been added to the list, it becomes the
   607         -** property of the writer thread and must not be read or modified by the
   608         -** caller.  
   609         -*/
   610         -static void addAsyncWrite(AsyncWrite *pWrite){
   611         -  /* We must hold the queue mutex in order to modify the queue pointers */
   612         -  if( pWrite->op!=ASYNC_UNLOCK ){
   613         -    pthread_mutex_lock(&async.queueMutex);
   614         -  }
   615         -
   616         -  /* Add the record to the end of the write-op queue */
   617         -  assert( !pWrite->pNext );
   618         -  if( async.pQueueLast ){
   619         -    assert( async.pQueueFirst );
   620         -    async.pQueueLast->pNext = pWrite;
   621         -  }else{
   622         -    async.pQueueFirst = pWrite;
   623         -  }
   624         -  async.pQueueLast = pWrite;
   625         -  ASYNC_TRACE(("PUSH %p (%s %s %d)\n", pWrite, azOpcodeName[pWrite->op],
   626         -         pWrite->pFileData ? pWrite->pFileData->zName : "-", pWrite->iOffset));
   627         -
   628         -  if( pWrite->op==ASYNC_CLOSE ){
   629         -    async.nFile--;
   630         -  }
   631         -
   632         -  /* Drop the queue mutex */
   633         -  if( pWrite->op!=ASYNC_UNLOCK ){
   634         -    pthread_mutex_unlock(&async.queueMutex);
   635         -  }
   636         -
   637         -  /* The writer thread might have been idle because there was nothing
   638         -  ** on the write-op queue for it to do.  So wake it up. */
   639         -  pthread_cond_signal(&async.queueSignal);
   640         -}
   641         -
   642         -/*
   643         -** Increment async.nFile in a thread-safe manner.
   644         -*/
   645         -static void incrOpenFileCount(void){
   646         -  /* We must hold the queue mutex in order to modify async.nFile */
   647         -  pthread_mutex_lock(&async.queueMutex);
   648         -  if( async.nFile==0 ){
   649         -    async.ioError = SQLITE_OK;
   650         -  }
   651         -  async.nFile++;
   652         -  pthread_mutex_unlock(&async.queueMutex);
   653         -}
   654         -
   655         -/*
   656         -** This is a utility function to allocate and populate a new AsyncWrite
   657         -** structure and insert it (via addAsyncWrite() ) into the global list.
   658         -*/
   659         -static int addNewAsyncWrite(
   660         -  AsyncFileData *pFileData, 
   661         -  int op, 
   662         -  sqlite3_int64 iOffset, 
   663         -  int nByte,
   664         -  const char *zByte
   665         -){
   666         -  AsyncWrite *p;
   667         -  if( op!=ASYNC_CLOSE && async.ioError ){
   668         -    return async.ioError;
   669         -  }
   670         -  p = sqlite3_malloc(sizeof(AsyncWrite) + (zByte?nByte:0));
   671         -  if( !p ){
   672         -    /* The upper layer does not expect operations like OsWrite() to
   673         -    ** return SQLITE_NOMEM. This is partly because under normal conditions
   674         -    ** SQLite is required to do rollback without calling malloc(). So
   675         -    ** if malloc() fails here, treat it as an I/O error. The above
   676         -    ** layer knows how to handle that.
   677         -    */
   678         -    return SQLITE_IOERR;
   679         -  }
   680         -  p->op = op;
   681         -  p->iOffset = iOffset;
   682         -  p->nByte = nByte;
   683         -  p->pFileData = pFileData;
   684         -  p->pNext = 0;
   685         -  if( zByte ){
   686         -    p->zBuf = (char *)&p[1];
   687         -    memcpy(p->zBuf, zByte, nByte);
   688         -  }else{
   689         -    p->zBuf = 0;
   690         -  }
   691         -  addAsyncWrite(p);
   692         -  return SQLITE_OK;
   693         -}
   694         -
   695         -/*
   696         -** Close the file. This just adds an entry to the write-op list, the file is
   697         -** not actually closed.
   698         -*/
   699         -static int asyncClose(sqlite3_file *pFile){
   700         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   701         -
   702         -  /* Unlock the file, if it is locked */
   703         -  pthread_mutex_lock(&async.lockMutex);
   704         -  p->lock.eLock = 0;
   705         -  pthread_mutex_unlock(&async.lockMutex);
   706         -
   707         -  addAsyncWrite(&p->closeOp);
   708         -  return SQLITE_OK;
   709         -}
   710         -
   711         -/*
   712         -** Implementation of sqlite3OsWrite() for asynchronous files. Instead of 
   713         -** writing to the underlying file, this function adds an entry to the end of
   714         -** the global AsyncWrite list. Either SQLITE_OK or SQLITE_NOMEM may be
   715         -** returned.
   716         -*/
   717         -static int asyncWrite(
   718         -  sqlite3_file *pFile, 
   719         -  const void *pBuf, 
   720         -  int amt, 
   721         -  sqlite3_int64 iOff
   722         -){
   723         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   724         -  return addNewAsyncWrite(p, ASYNC_WRITE, iOff, amt, pBuf);
   725         -}
   726         -
   727         -/*
   728         -** Read data from the file. First we read from the filesystem, then adjust 
   729         -** the contents of the buffer based on ASYNC_WRITE operations in the 
   730         -** write-op queue.
   731         -**
   732         -** This method holds the mutex from start to finish.
   733         -*/
   734         -static int asyncRead(
   735         -  sqlite3_file *pFile, 
   736         -  void *zOut, 
   737         -  int iAmt, 
   738         -  sqlite3_int64 iOffset
   739         -){
   740         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   741         -  int rc = SQLITE_OK;
   742         -  sqlite3_int64 filesize;
   743         -  int nRead;
   744         -  sqlite3_file *pBase = p->pBaseRead;
   745         -
   746         -  /* Grab the write queue mutex for the duration of the call */
   747         -  pthread_mutex_lock(&async.queueMutex);
   748         -
   749         -  /* If an I/O error has previously occurred in this virtual file 
   750         -  ** system, then all subsequent operations fail.
   751         -  */
   752         -  if( async.ioError!=SQLITE_OK ){
   753         -    rc = async.ioError;
   754         -    goto asyncread_out;
   755         -  }
   756         -
   757         -  if( pBase->pMethods ){
   758         -    rc = pBase->pMethods->xFileSize(pBase, &filesize);
   759         -    if( rc!=SQLITE_OK ){
   760         -      goto asyncread_out;
   761         -    }
   762         -    nRead = MIN(filesize - iOffset, iAmt);
   763         -    if( nRead>0 ){
   764         -      rc = pBase->pMethods->xRead(pBase, zOut, nRead, iOffset);
   765         -      ASYNC_TRACE(("READ %s %d bytes at %d\n", p->zName, nRead, iOffset));
   766         -    }
   767         -  }
   768         -
   769         -  if( rc==SQLITE_OK ){
   770         -    AsyncWrite *pWrite;
   771         -    char *zName = p->zName;
   772         -
   773         -    for(pWrite=async.pQueueFirst; pWrite; pWrite = pWrite->pNext){
   774         -      if( pWrite->op==ASYNC_WRITE && (
   775         -        (pWrite->pFileData==p) ||
   776         -        (zName && pWrite->pFileData->zName==zName)
   777         -      )){
   778         -        int iBeginOut = (pWrite->iOffset-iOffset);
   779         -        int iBeginIn = -iBeginOut;
   780         -        int nCopy;
   781         -
   782         -        if( iBeginIn<0 ) iBeginIn = 0;
   783         -        if( iBeginOut<0 ) iBeginOut = 0;
   784         -        nCopy = MIN(pWrite->nByte-iBeginIn, iAmt-iBeginOut);
   785         -
   786         -        if( nCopy>0 ){
   787         -          memcpy(&((char *)zOut)[iBeginOut], &pWrite->zBuf[iBeginIn], nCopy);
   788         -          ASYNC_TRACE(("OVERREAD %d bytes at %d\n", nCopy, iBeginOut+iOffset));
   789         -        }
   790         -      }
   791         -    }
   792         -  }
   793         -
   794         -asyncread_out:
   795         -  pthread_mutex_unlock(&async.queueMutex);
   796         -  return rc;
   797         -}
   798         -
   799         -/*
   800         -** Truncate the file to nByte bytes in length. This just adds an entry to 
   801         -** the write-op list, no IO actually takes place.
   802         -*/
   803         -static int asyncTruncate(sqlite3_file *pFile, sqlite3_int64 nByte){
   804         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   805         -  return addNewAsyncWrite(p, ASYNC_TRUNCATE, nByte, 0, 0);
   806         -}
   807         -
   808         -/*
   809         -** Sync the file. This just adds an entry to the write-op list, the 
   810         -** sync() is done later by sqlite3_async_flush().
   811         -*/
   812         -static int asyncSync(sqlite3_file *pFile, int flags){
   813         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   814         -  return addNewAsyncWrite(p, ASYNC_SYNC, 0, flags, 0);
   815         -}
   816         -
   817         -/*
   818         -** Read the size of the file. First we read the size of the file system 
   819         -** entry, then adjust for any ASYNC_WRITE or ASYNC_TRUNCATE operations 
   820         -** currently in the write-op list. 
   821         -**
   822         -** This method holds the mutex from start to finish.
   823         -*/
   824         -int asyncFileSize(sqlite3_file *pFile, sqlite3_int64 *piSize){
   825         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   826         -  int rc = SQLITE_OK;
   827         -  sqlite3_int64 s = 0;
   828         -  sqlite3_file *pBase;
   829         -
   830         -  pthread_mutex_lock(&async.queueMutex);
   831         -
   832         -  /* Read the filesystem size from the base file. If pBaseRead is NULL, this
   833         -  ** means the file hasn't been opened yet. In this case all relevant data 
   834         -  ** must be in the write-op queue anyway, so we can omit reading from the
   835         -  ** file-system.
   836         -  */
   837         -  pBase = p->pBaseRead;
   838         -  if( pBase->pMethods ){
   839         -    rc = pBase->pMethods->xFileSize(pBase, &s);
   840         -  }
   841         -
   842         -  if( rc==SQLITE_OK ){
   843         -    AsyncWrite *pWrite;
   844         -    for(pWrite=async.pQueueFirst; pWrite; pWrite = pWrite->pNext){
   845         -      if( pWrite->op==ASYNC_DELETE 
   846         -       && p->zName 
   847         -       && strcmp(p->zName, pWrite->zBuf)==0 
   848         -      ){
   849         -        s = 0;
   850         -      }else if( pWrite->pFileData && (
   851         -          (pWrite->pFileData==p) 
   852         -       || (p->zName && pWrite->pFileData->zName==p->zName) 
   853         -      )){
   854         -        switch( pWrite->op ){
   855         -          case ASYNC_WRITE:
   856         -            s = MAX(pWrite->iOffset + (sqlite3_int64)(pWrite->nByte), s);
   857         -            break;
   858         -          case ASYNC_TRUNCATE:
   859         -            s = MIN(s, pWrite->iOffset);
   860         -            break;
   861         -        }
   862         -      }
   863         -    }
   864         -    *piSize = s;
   865         -  }
   866         -  pthread_mutex_unlock(&async.queueMutex);
   867         -  return rc;
   868         -}
   869         -
   870         -/*
   871         -** Lock or unlock the actual file-system entry.
   872         -*/
   873         -static int getFileLock(AsyncLock *pLock){
   874         -  int rc = SQLITE_OK;
   875         -  AsyncFileLock *pIter;
   876         -  int eRequired = 0;
   877         -
   878         -  if( pLock->pFile ){
   879         -    for(pIter=pLock->pList; pIter; pIter=pIter->pNext){
   880         -      assert(pIter->eAsyncLock>=pIter->eLock);
   881         -      if( pIter->eAsyncLock>eRequired ){
   882         -        eRequired = pIter->eAsyncLock;
   883         -        assert(eRequired>=0 && eRequired<=SQLITE_LOCK_EXCLUSIVE);
   884         -      }
   885         -    }
   886         -
   887         -    if( eRequired>pLock->eLock ){
   888         -      rc = pLock->pFile->pMethods->xLock(pLock->pFile, eRequired);
   889         -      if( rc==SQLITE_OK ){
   890         -        pLock->eLock = eRequired;
   891         -      }
   892         -    }
   893         -    else if( eRequired<pLock->eLock && eRequired<=SQLITE_LOCK_SHARED ){
   894         -      rc = pLock->pFile->pMethods->xUnlock(pLock->pFile, eRequired);
   895         -      if( rc==SQLITE_OK ){
   896         -        pLock->eLock = eRequired;
   897         -      }
   898         -    }
   899         -  }
   900         -
   901         -  return rc;
   902         -}
   903         -
   904         -/*
   905         -** Return the AsyncLock structure from the global async.pLock list 
   906         -** associated with the file-system entry identified by path zName 
   907         -** (a string of nName bytes). If no such structure exists, return 0.
   908         -*/
   909         -static AsyncLock *findLock(const char *zName, int nName){
   910         -  AsyncLock *p = async.pLock;
   911         -  while( p && (p->nFile!=nName || memcmp(p->zFile, zName, nName)) ){
   912         -    p = p->pNext;
   913         -  }
   914         -  return p;
   915         -}
   916         -
   917         -/*
   918         -** The following two methods - asyncLock() and asyncUnlock() - are used
   919         -** to obtain and release locks on database files opened with the
   920         -** asynchronous backend.
   921         -*/
   922         -static int asyncLock(sqlite3_file *pFile, int eLock){
   923         -  int rc = SQLITE_OK;
   924         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   925         -
   926         -  if( p->zName ){
   927         -    pthread_mutex_lock(&async.lockMutex);
   928         -    if( p->lock.eLock<eLock ){
   929         -      AsyncLock *pLock = p->pLock;
   930         -      AsyncFileLock *pIter;
   931         -      assert(pLock && pLock->pList);
   932         -      for(pIter=pLock->pList; pIter; pIter=pIter->pNext){
   933         -        if( pIter!=&p->lock && (
   934         -          (eLock==SQLITE_LOCK_EXCLUSIVE && pIter->eLock>=SQLITE_LOCK_SHARED) ||
   935         -          (eLock==SQLITE_LOCK_PENDING && pIter->eLock>=SQLITE_LOCK_RESERVED) ||
   936         -          (eLock==SQLITE_LOCK_RESERVED && pIter->eLock>=SQLITE_LOCK_RESERVED) ||
   937         -          (eLock==SQLITE_LOCK_SHARED && pIter->eLock>=SQLITE_LOCK_PENDING)
   938         -        )){
   939         -          rc = SQLITE_BUSY;
   940         -        }
   941         -      }
   942         -      if( rc==SQLITE_OK ){
   943         -        p->lock.eLock = eLock;
   944         -        p->lock.eAsyncLock = MAX(p->lock.eAsyncLock, eLock);
   945         -      }
   946         -      assert(p->lock.eAsyncLock>=p->lock.eLock);
   947         -      if( rc==SQLITE_OK ){
   948         -        rc = getFileLock(pLock);
   949         -      }
   950         -    }
   951         -    pthread_mutex_unlock(&async.lockMutex);
   952         -  }
   953         -
   954         -  ASYNC_TRACE(("LOCK %d (%s) rc=%d\n", eLock, p->zName, rc));
   955         -  return rc;
   956         -}
   957         -static int asyncUnlock(sqlite3_file *pFile, int eLock){
   958         -  int rc = SQLITE_OK;
   959         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   960         -  if( p->zName ){
   961         -    AsyncFileLock *pLock = &p->lock;
   962         -    pthread_mutex_lock(&async.queueMutex);
   963         -    pthread_mutex_lock(&async.lockMutex);
   964         -    pLock->eLock = MIN(pLock->eLock, eLock);
   965         -    rc = addNewAsyncWrite(p, ASYNC_UNLOCK, 0, eLock, 0);
   966         -    pthread_mutex_unlock(&async.lockMutex);
   967         -    pthread_mutex_unlock(&async.queueMutex);
   968         -  }
   969         -  return rc;
   970         -}
   971         -
   972         -/*
   973         -** This function is called when the pager layer first opens a database file
   974         -** and is checking for a hot-journal.
   975         -*/
   976         -static int asyncCheckReservedLock(sqlite3_file *pFile, int *pResOut){
   977         -  int ret = 0;
   978         -  AsyncFileLock *pIter;
   979         -  AsyncFileData *p = ((AsyncFile *)pFile)->pData;
   980         -
   981         -  pthread_mutex_lock(&async.lockMutex);
   982         -  for(pIter=p->pLock->pList; pIter; pIter=pIter->pNext){
   983         -    if( pIter->eLock>=SQLITE_LOCK_RESERVED ){
   984         -      ret = 1;
   985         -    }
   986         -  }
   987         -  pthread_mutex_unlock(&async.lockMutex);
   988         -
   989         -  ASYNC_TRACE(("CHECK-LOCK %d (%s)\n", ret, p->zName));
   990         -  *pResOut = ret;
   991         -  return SQLITE_OK;
   992         -}
   993         -
   994         -/* 
   995         -** sqlite3_file_control() implementation.
   996         -*/
   997         -static int asyncFileControl(sqlite3_file *id, int op, void *pArg){
   998         -  switch( op ){
   999         -    case SQLITE_FCNTL_LOCKSTATE: {
  1000         -      pthread_mutex_lock(&async.lockMutex);
  1001         -      *(int*)pArg = ((AsyncFile*)id)->pData->lock.eLock;
  1002         -      pthread_mutex_unlock(&async.lockMutex);
  1003         -      return SQLITE_OK;
  1004         -    }
  1005         -  }
  1006         -  return SQLITE_ERROR;
  1007         -}
  1008         -
  1009         -/* 
  1010         -** Return the device characteristics and sector-size of the device. It
  1011         -** is not tricky to implement these correctly, as this backend might 
  1012         -** not have an open file handle at this point.
  1013         -*/
  1014         -static int asyncSectorSize(sqlite3_file *pFile){
  1015         -  return 512;
  1016         -}
  1017         -static int asyncDeviceCharacteristics(sqlite3_file *pFile){
  1018         -  return 0;
  1019         -}
  1020         -
  1021         -static int unlinkAsyncFile(AsyncFileData *pData){
  1022         -  AsyncFileLock **ppIter;
  1023         -  int rc = SQLITE_OK;
  1024         -
  1025         -  if( pData->zName ){
  1026         -    AsyncLock *pLock = pData->pLock;
  1027         -    for(ppIter=&pLock->pList; *ppIter; ppIter=&((*ppIter)->pNext)){
  1028         -      if( (*ppIter)==&pData->lock ){
  1029         -        *ppIter = pData->lock.pNext;
  1030         -        break;
  1031         -      }
  1032         -    }
  1033         -    if( !pLock->pList ){
  1034         -      AsyncLock **pp;
  1035         -      if( pLock->pFile ){
  1036         -        pLock->pFile->pMethods->xClose(pLock->pFile);
  1037         -      }
  1038         -      for(pp=&async.pLock; *pp!=pLock; pp=&((*pp)->pNext));
  1039         -      *pp = pLock->pNext;
  1040         -      sqlite3_free(pLock);
  1041         -    }else{
  1042         -      rc = getFileLock(pLock);
  1043         -    }
  1044         -  }
  1045         -
  1046         -  return rc;
  1047         -}
  1048         -
  1049         -/*
  1050         -** The parameter passed to this function is a copy of a 'flags' parameter
  1051         -** passed to this modules xOpen() method. This function returns true
  1052         -** if the file should be opened asynchronously, or false if it should
  1053         -** be opened immediately.
  1054         -**
  1055         -** If the file is to be opened asynchronously, then asyncOpen() will add
  1056         -** an entry to the event queue and the file will not actually be opened
  1057         -** until the event is processed. Otherwise, the file is opened directly
  1058         -** by the caller.
  1059         -*/
  1060         -static int doAsynchronousOpen(int flags){
  1061         -  return (flags&SQLITE_OPEN_CREATE) && (
  1062         -      (flags&SQLITE_OPEN_MAIN_JOURNAL) ||
  1063         -      (flags&SQLITE_OPEN_TEMP_JOURNAL) ||
  1064         -      (flags&SQLITE_OPEN_DELETEONCLOSE)
  1065         -  );
  1066         -}
  1067         -
  1068         -/*
  1069         -** Open a file.
  1070         -*/
  1071         -static int asyncOpen(
  1072         -  sqlite3_vfs *pAsyncVfs,
  1073         -  const char *zName,
  1074         -  sqlite3_file *pFile,
  1075         -  int flags,
  1076         -  int *pOutFlags
  1077         -){
  1078         -  static sqlite3_io_methods async_methods = {
  1079         -    1,                               /* iVersion */
  1080         -    asyncClose,                      /* xClose */
  1081         -    asyncRead,                       /* xRead */
  1082         -    asyncWrite,                      /* xWrite */
  1083         -    asyncTruncate,                   /* xTruncate */
  1084         -    asyncSync,                       /* xSync */
  1085         -    asyncFileSize,                   /* xFileSize */
  1086         -    asyncLock,                       /* xLock */
  1087         -    asyncUnlock,                     /* xUnlock */
  1088         -    asyncCheckReservedLock,          /* xCheckReservedLock */
  1089         -    asyncFileControl,                /* xFileControl */
  1090         -    asyncSectorSize,                 /* xSectorSize */
  1091         -    asyncDeviceCharacteristics       /* xDeviceCharacteristics */
  1092         -  };
  1093         -
  1094         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1095         -  AsyncFile *p = (AsyncFile *)pFile;
  1096         -  int nName = 0;
  1097         -  int rc = SQLITE_OK;
  1098         -  int nByte;
  1099         -  AsyncFileData *pData;
  1100         -  AsyncLock *pLock = 0;
  1101         -  char *z;
  1102         -  int isAsyncOpen = doAsynchronousOpen(flags);
  1103         -
  1104         -  /* If zName is NULL, then the upper layer is requesting an anonymous file */
  1105         -  if( zName ){
  1106         -    nName = strlen(zName)+1;
  1107         -  }
  1108         -
  1109         -  nByte = (
  1110         -    sizeof(AsyncFileData) +        /* AsyncFileData structure */
  1111         -    2 * pVfs->szOsFile +           /* AsyncFileData.pBaseRead and pBaseWrite */
  1112         -    nName                          /* AsyncFileData.zName */
  1113         -  ); 
  1114         -  z = sqlite3_malloc(nByte);
  1115         -  if( !z ){
  1116         -    return SQLITE_NOMEM;
  1117         -  }
  1118         -  memset(z, 0, nByte);
  1119         -  pData = (AsyncFileData*)z;
  1120         -  z += sizeof(pData[0]);
  1121         -  pData->pBaseRead = (sqlite3_file*)z;
  1122         -  z += pVfs->szOsFile;
  1123         -  pData->pBaseWrite = (sqlite3_file*)z;
  1124         -  pData->closeOp.pFileData = pData;
  1125         -  pData->closeOp.op = ASYNC_CLOSE;
  1126         -
  1127         -  if( zName ){
  1128         -    z += pVfs->szOsFile;
  1129         -    pData->zName = z;
  1130         -    pData->nName = nName;
  1131         -    memcpy(pData->zName, zName, nName);
  1132         -  }
  1133         -
  1134         -  if( !isAsyncOpen ){
  1135         -    int flagsout;
  1136         -    rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseRead, flags, &flagsout);
  1137         -    if( rc==SQLITE_OK && (flagsout&SQLITE_OPEN_READWRITE) ){
  1138         -      rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseWrite, flags, 0);
  1139         -    }
  1140         -    if( pOutFlags ){
  1141         -      *pOutFlags = flagsout;
  1142         -    }
  1143         -  }
  1144         -
  1145         -  pthread_mutex_lock(&async.lockMutex);
  1146         -
  1147         -  if( zName && rc==SQLITE_OK ){
  1148         -    pLock = findLock(pData->zName, pData->nName);
  1149         -    if( !pLock ){
  1150         -      int nByte = pVfs->szOsFile + sizeof(AsyncLock) + pData->nName + 1; 
  1151         -      pLock = (AsyncLock *)sqlite3_malloc(nByte);
  1152         -      if( pLock ){
  1153         -        memset(pLock, 0, nByte);
  1154         -#ifdef ENABLE_FILE_LOCKING
  1155         -        if( flags&SQLITE_OPEN_MAIN_DB ){
  1156         -          pLock->pFile = (sqlite3_file *)&pLock[1];
  1157         -          rc = pVfs->xOpen(pVfs, pData->zName, pLock->pFile, flags, 0);
  1158         -          if( rc!=SQLITE_OK ){
  1159         -            sqlite3_free(pLock);
  1160         -            pLock = 0;
  1161         -          }
  1162         -        }
  1163         -#endif
  1164         -        if( pLock ){
  1165         -          pLock->nFile = pData->nName;
  1166         -          pLock->zFile = &((char *)(&pLock[1]))[pVfs->szOsFile];
  1167         -          memcpy(pLock->zFile, pData->zName, pLock->nFile);
  1168         -          pLock->pNext = async.pLock;
  1169         -          async.pLock = pLock;
  1170         -        }
  1171         -      }else{
  1172         -        rc = SQLITE_NOMEM;
  1173         -      }
  1174         -    }
  1175         -  }
  1176         -
  1177         -  if( rc==SQLITE_OK ){
  1178         -    p->pMethod = &async_methods;
  1179         -    p->pData = pData;
  1180         -
  1181         -    /* Link AsyncFileData.lock into the linked list of 
  1182         -    ** AsyncFileLock structures for this file.
  1183         -    */
  1184         -    if( zName ){
  1185         -      pData->lock.pNext = pLock->pList;
  1186         -      pLock->pList = &pData->lock;
  1187         -      pData->zName = pLock->zFile;
  1188         -    }
  1189         -  }else{
  1190         -    if( pData->pBaseRead->pMethods ){
  1191         -      pData->pBaseRead->pMethods->xClose(pData->pBaseRead);
  1192         -    }
  1193         -    if( pData->pBaseWrite->pMethods ){
  1194         -      pData->pBaseWrite->pMethods->xClose(pData->pBaseWrite);
  1195         -    }
  1196         -    sqlite3_free(pData);
  1197         -  }
  1198         -
  1199         -  pthread_mutex_unlock(&async.lockMutex);
  1200         -
  1201         -  if( rc==SQLITE_OK ){
  1202         -    incrOpenFileCount();
  1203         -    pData->pLock = pLock;
  1204         -  }
  1205         -
  1206         -  if( rc==SQLITE_OK && isAsyncOpen ){
  1207         -    rc = addNewAsyncWrite(pData, ASYNC_OPENEXCLUSIVE, (sqlite3_int64)flags,0,0);
  1208         -    if( rc==SQLITE_OK ){
  1209         -      if( pOutFlags ) *pOutFlags = flags;
  1210         -    }else{
  1211         -      pthread_mutex_lock(&async.lockMutex);
  1212         -      unlinkAsyncFile(pData);
  1213         -      pthread_mutex_unlock(&async.lockMutex);
  1214         -      sqlite3_free(pData);
  1215         -    }
  1216         -  }
  1217         -  if( rc!=SQLITE_OK ){
  1218         -    p->pMethod = 0;
  1219         -  }
  1220         -  return rc;
  1221         -}
  1222         -
  1223         -/*
  1224         -** Implementation of sqlite3OsDelete. Add an entry to the end of the 
  1225         -** write-op queue to perform the delete.
  1226         -*/
  1227         -static int asyncDelete(sqlite3_vfs *pAsyncVfs, const char *z, int syncDir){
  1228         -  return addNewAsyncWrite(0, ASYNC_DELETE, syncDir, strlen(z)+1, z);
  1229         -}
  1230         -
  1231         -/*
  1232         -** Implementation of sqlite3OsAccess. This method holds the mutex from
  1233         -** start to finish.
  1234         -*/
  1235         -static int asyncAccess(
  1236         -  sqlite3_vfs *pAsyncVfs, 
  1237         -  const char *zName, 
  1238         -  int flags,
  1239         -  int *pResOut
  1240         -){
  1241         -  int rc;
  1242         -  int ret;
  1243         -  AsyncWrite *p;
  1244         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1245         -
  1246         -  assert(flags==SQLITE_ACCESS_READWRITE 
  1247         -      || flags==SQLITE_ACCESS_READ 
  1248         -      || flags==SQLITE_ACCESS_EXISTS 
  1249         -  );
  1250         -
  1251         -  pthread_mutex_lock(&async.queueMutex);
  1252         -  rc = pVfs->xAccess(pVfs, zName, flags, &ret);
  1253         -  if( rc==SQLITE_OK && flags==SQLITE_ACCESS_EXISTS ){
  1254         -    for(p=async.pQueueFirst; p; p = p->pNext){
  1255         -      if( p->op==ASYNC_DELETE && 0==strcmp(p->zBuf, zName) ){
  1256         -        ret = 0;
  1257         -      }else if( p->op==ASYNC_OPENEXCLUSIVE 
  1258         -             && p->pFileData->zName
  1259         -             && 0==strcmp(p->pFileData->zName, zName) 
  1260         -      ){
  1261         -        ret = 1;
  1262         -      }
  1263         -    }
  1264         -  }
  1265         -  ASYNC_TRACE(("ACCESS(%s): %s = %d\n", 
  1266         -    flags==SQLITE_ACCESS_READWRITE?"read-write":
  1267         -    flags==SQLITE_ACCESS_READ?"read":"exists"
  1268         -    , zName, ret)
  1269         -  );
  1270         -  pthread_mutex_unlock(&async.queueMutex);
  1271         -  *pResOut = ret;
  1272         -  return rc;
  1273         -}
  1274         -
  1275         -/*
  1276         -** Fill in zPathOut with the full path to the file identified by zPath.
  1277         -*/
  1278         -static int asyncFullPathname(
  1279         -  sqlite3_vfs *pAsyncVfs, 
  1280         -  const char *zPath, 
  1281         -  int nPathOut,
  1282         -  char *zPathOut
  1283         -){
  1284         -  int rc;
  1285         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1286         -  rc = pVfs->xFullPathname(pVfs, zPath, nPathOut, zPathOut);
  1287         -
  1288         -  /* Because of the way intra-process file locking works, this backend
  1289         -  ** needs to return a canonical path. The following block assumes the
  1290         -  ** file-system uses unix style paths. 
  1291         -  */
  1292         -  if( rc==SQLITE_OK ){
  1293         -    int i, j;
  1294         -    int n = nPathOut;
  1295         -    char *z = zPathOut;
  1296         -    while( n>1 && z[n-1]=='/' ){ n--; }
  1297         -    for(i=j=0; i<n; i++){
  1298         -      if( z[i]=='/' ){
  1299         -        if( z[i+1]=='/' ) continue;
  1300         -        if( z[i+1]=='.' && i+2<n && z[i+2]=='/' ){
  1301         -          i += 1;
  1302         -          continue;
  1303         -        }
  1304         -        if( z[i+1]=='.' && i+3<n && z[i+2]=='.' && z[i+3]=='/' ){
  1305         -          while( j>0 && z[j-1]!='/' ){ j--; }
  1306         -          if( j>0 ){ j--; }
  1307         -          i += 2;
  1308         -          continue;
  1309         -        }
  1310         -      }
  1311         -      z[j++] = z[i];
  1312         -    }
  1313         -    z[j] = 0;
  1314         -  }
  1315         -
  1316         -  return rc;
  1317         -}
  1318         -static void *asyncDlOpen(sqlite3_vfs *pAsyncVfs, const char *zPath){
  1319         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1320         -  return pVfs->xDlOpen(pVfs, zPath);
  1321         -}
  1322         -static void asyncDlError(sqlite3_vfs *pAsyncVfs, int nByte, char *zErrMsg){
  1323         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1324         -  pVfs->xDlError(pVfs, nByte, zErrMsg);
  1325         -}
  1326         -static void (*asyncDlSym(
  1327         -  sqlite3_vfs *pAsyncVfs, 
  1328         -  void *pHandle, 
  1329         -  const char *zSymbol
  1330         -))(void){
  1331         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1332         -  return pVfs->xDlSym(pVfs, pHandle, zSymbol);
  1333         -}
  1334         -static void asyncDlClose(sqlite3_vfs *pAsyncVfs, void *pHandle){
  1335         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1336         -  pVfs->xDlClose(pVfs, pHandle);
  1337         -}
  1338         -static int asyncRandomness(sqlite3_vfs *pAsyncVfs, int nByte, char *zBufOut){
  1339         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1340         -  return pVfs->xRandomness(pVfs, nByte, zBufOut);
  1341         -}
  1342         -static int asyncSleep(sqlite3_vfs *pAsyncVfs, int nMicro){
  1343         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1344         -  return pVfs->xSleep(pVfs, nMicro);
  1345         -}
  1346         -static int asyncCurrentTime(sqlite3_vfs *pAsyncVfs, double *pTimeOut){
  1347         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)pAsyncVfs->pAppData;
  1348         -  return pVfs->xCurrentTime(pVfs, pTimeOut);
  1349         -}
  1350         -
  1351         -static sqlite3_vfs async_vfs = {
  1352         -  1,                    /* iVersion */
  1353         -  sizeof(AsyncFile),    /* szOsFile */
  1354         -  0,                    /* mxPathname */
  1355         -  0,                    /* pNext */
  1356         -  "async",              /* zName */
  1357         -  0,                    /* pAppData */
  1358         -  asyncOpen,            /* xOpen */
  1359         -  asyncDelete,          /* xDelete */
  1360         -  asyncAccess,          /* xAccess */
  1361         -  asyncFullPathname,    /* xFullPathname */
  1362         -  asyncDlOpen,          /* xDlOpen */
  1363         -  asyncDlError,         /* xDlError */
  1364         -  asyncDlSym,           /* xDlSym */
  1365         -  asyncDlClose,         /* xDlClose */
  1366         -  asyncRandomness,      /* xDlError */
  1367         -  asyncSleep,           /* xDlSym */
  1368         -  asyncCurrentTime      /* xDlClose */
  1369         -};
  1370         -
  1371         -/*
  1372         -** Call this routine to enable or disable the
  1373         -** asynchronous IO features implemented in this file. 
  1374         -**
  1375         -** This routine is not even remotely threadsafe.  Do not call
  1376         -** this routine while any SQLite database connections are open.
  1377         -*/
  1378         -static void asyncEnable(int enable){
  1379         -  if( enable ){
  1380         -    if( !async_vfs.pAppData ){
  1381         -      async_vfs.pAppData = (void *)sqlite3_vfs_find(0);
  1382         -      async_vfs.mxPathname = ((sqlite3_vfs *)async_vfs.pAppData)->mxPathname;
  1383         -      sqlite3_vfs_register(&async_vfs, 1);
  1384         -    }
  1385         -  }else{
  1386         -    if( async_vfs.pAppData ){
  1387         -      sqlite3_vfs_unregister(&async_vfs);
  1388         -      async_vfs.pAppData = 0;
  1389         -    }
  1390         -  }
  1391         -}
  1392         -
  1393         -/* 
  1394         -** This procedure runs in a separate thread, reading messages off of the
  1395         -** write queue and processing them one by one.  
  1396         -**
  1397         -** If async.writerHaltNow is true, then this procedure exits
  1398         -** after processing a single message.
  1399         -**
  1400         -** If async.writerHaltWhenIdle is true, then this procedure exits when
  1401         -** the write queue is empty.
  1402         -**
  1403         -** If both of the above variables are false, this procedure runs
  1404         -** indefinately, waiting for operations to be added to the write queue
  1405         -** and processing them in the order in which they arrive.
  1406         -**
  1407         -** An artifical delay of async.ioDelay milliseconds is inserted before
  1408         -** each write operation in order to simulate the effect of a slow disk.
  1409         -**
  1410         -** Only one instance of this procedure may be running at a time.
  1411         -*/
  1412         -static void *asyncWriterThread(void *pIsStarted){
  1413         -  sqlite3_vfs *pVfs = (sqlite3_vfs *)(async_vfs.pAppData);
  1414         -  AsyncWrite *p = 0;
  1415         -  int rc = SQLITE_OK;
  1416         -  int holdingMutex = 0;
  1417         -
  1418         -  if( pthread_mutex_trylock(&async.writerMutex) ){
  1419         -    return 0;
  1420         -  }
  1421         -  (*(int *)pIsStarted) = 1;
  1422         -  while( async.writerHaltNow==0 ){
  1423         -    int doNotFree = 0;
  1424         -    sqlite3_file *pBase = 0;
  1425         -
  1426         -    if( !holdingMutex ){
  1427         -      pthread_mutex_lock(&async.queueMutex);
  1428         -    }
  1429         -    while( (p = async.pQueueFirst)==0 ){
  1430         -      pthread_cond_broadcast(&async.emptySignal);
  1431         -      if( async.writerHaltWhenIdle ){
  1432         -        pthread_mutex_unlock(&async.queueMutex);
  1433         -        break;
  1434         -      }else{
  1435         -        ASYNC_TRACE(("IDLE\n"));
  1436         -        pthread_cond_wait(&async.queueSignal, &async.queueMutex);
  1437         -        ASYNC_TRACE(("WAKEUP\n"));
  1438         -      }
  1439         -    }
  1440         -    if( p==0 ) break;
  1441         -    holdingMutex = 1;
  1442         -
  1443         -    /* Right now this thread is holding the mutex on the write-op queue.
  1444         -    ** Variable 'p' points to the first entry in the write-op queue. In
  1445         -    ** the general case, we hold on to the mutex for the entire body of
  1446         -    ** the loop. 
  1447         -    **
  1448         -    ** However in the cases enumerated below, we relinquish the mutex,
  1449         -    ** perform the IO, and then re-request the mutex before removing 'p' from
  1450         -    ** the head of the write-op queue. The idea is to increase concurrency with
  1451         -    ** sqlite threads.
  1452         -    **
  1453         -    **     * An ASYNC_CLOSE operation.
  1454         -    **     * An ASYNC_OPENEXCLUSIVE operation. For this one, we relinquish 
  1455         -    **       the mutex, call the underlying xOpenExclusive() function, then
  1456         -    **       re-aquire the mutex before seting the AsyncFile.pBaseRead 
  1457         -    **       variable.
  1458         -    **     * ASYNC_SYNC and ASYNC_WRITE operations, if 
  1459         -    **       SQLITE_ASYNC_TWO_FILEHANDLES was set at compile time and two
  1460         -    **       file-handles are open for the particular file being "synced".
  1461         -    */
  1462         -    if( async.ioError!=SQLITE_OK && p->op!=ASYNC_CLOSE ){
  1463         -      p->op = ASYNC_NOOP;
  1464         -    }
  1465         -    if( p->pFileData ){
  1466         -      pBase = p->pFileData->pBaseWrite;
  1467         -      if( 
  1468         -        p->op==ASYNC_CLOSE || 
  1469         -        p->op==ASYNC_OPENEXCLUSIVE ||
  1470         -        (pBase->pMethods && (p->op==ASYNC_SYNC || p->op==ASYNC_WRITE) ) 
  1471         -      ){
  1472         -        pthread_mutex_unlock(&async.queueMutex);
  1473         -        holdingMutex = 0;
  1474         -      }
  1475         -      if( !pBase->pMethods ){
  1476         -        pBase = p->pFileData->pBaseRead;
  1477         -      }
  1478         -    }
  1479         -
  1480         -    switch( p->op ){
  1481         -      case ASYNC_NOOP:
  1482         -        break;
  1483         -
  1484         -      case ASYNC_WRITE:
  1485         -        assert( pBase );
  1486         -        ASYNC_TRACE(("WRITE %s %d bytes at %d\n",
  1487         -                p->pFileData->zName, p->nByte, p->iOffset));
  1488         -        rc = pBase->pMethods->xWrite(pBase, (void *)(p->zBuf), p->nByte, p->iOffset);
  1489         -        break;
  1490         -
  1491         -      case ASYNC_SYNC:
  1492         -        assert( pBase );
  1493         -        ASYNC_TRACE(("SYNC %s\n", p->pFileData->zName));
  1494         -        rc = pBase->pMethods->xSync(pBase, p->nByte);
  1495         -        break;
  1496         -
  1497         -      case ASYNC_TRUNCATE:
  1498         -        assert( pBase );
  1499         -        ASYNC_TRACE(("TRUNCATE %s to %d bytes\n", 
  1500         -                p->pFileData->zName, p->iOffset));
  1501         -        rc = pBase->pMethods->xTruncate(pBase, p->iOffset);
  1502         -        break;
  1503         -
  1504         -      case ASYNC_CLOSE: {
  1505         -        AsyncFileData *pData = p->pFileData;
  1506         -        ASYNC_TRACE(("CLOSE %s\n", p->pFileData->zName));
  1507         -        if( pData->pBaseWrite->pMethods ){
  1508         -          pData->pBaseWrite->pMethods->xClose(pData->pBaseWrite);
  1509         -        }
  1510         -        if( pData->pBaseRead->pMethods ){
  1511         -          pData->pBaseRead->pMethods->xClose(pData->pBaseRead);
  1512         -        }
  1513         -
  1514         -        /* Unlink AsyncFileData.lock from the linked list of AsyncFileLock 
  1515         -        ** structures for this file. Obtain the async.lockMutex mutex 
  1516         -        ** before doing so.
  1517         -        */
  1518         -        pthread_mutex_lock(&async.lockMutex);
  1519         -        rc = unlinkAsyncFile(pData);
  1520         -        pthread_mutex_unlock(&async.lockMutex);
  1521         -
  1522         -        if( !holdingMutex ){
  1523         -          pthread_mutex_lock(&async.queueMutex);
  1524         -          holdingMutex = 1;
  1525         -        }
  1526         -        assert_mutex_is_held(&async.queueMutex);
  1527         -        async.pQueueFirst = p->pNext;
  1528         -        sqlite3_free(pData);
  1529         -        doNotFree = 1;
  1530         -        break;
  1531         -      }
  1532         -
  1533         -      case ASYNC_UNLOCK: {
  1534         -        AsyncWrite *pIter;
  1535         -        AsyncFileData *pData = p->pFileData;
  1536         -        int eLock = p->nByte;
  1537         -
  1538         -        /* When a file is locked by SQLite using the async backend, it is 
  1539         -        ** locked within the 'real' file-system synchronously. When it is
  1540         -        ** unlocked, an ASYNC_UNLOCK event is added to the write-queue to
  1541         -        ** unlock the file asynchronously. The design of the async backend
  1542         -        ** requires that the 'real' file-system file be locked from the
  1543         -        ** time that SQLite first locks it (and probably reads from it)
  1544         -        ** until all asynchronous write events that were scheduled before
  1545         -        ** SQLite unlocked the file have been processed.
  1546         -        **
  1547         -        ** This is more complex if SQLite locks and unlocks the file multiple
  1548         -        ** times in quick succession. For example, if SQLite does: 
  1549         -        ** 
  1550         -        **   lock, write, unlock, lock, write, unlock
  1551         -        **
  1552         -        ** Each "lock" operation locks the file immediately. Each "write" 
  1553         -        ** and "unlock" operation adds an event to the event queue. If the
  1554         -        ** second "lock" operation is performed before the first "unlock"
  1555         -        ** operation has been processed asynchronously, then the first
  1556         -        ** "unlock" cannot be safely processed as is, since this would mean
  1557         -        ** the file was unlocked when the second "write" operation is
  1558         -        ** processed. To work around this, when processing an ASYNC_UNLOCK
  1559         -        ** operation, SQLite:
  1560         -        **
  1561         -        **   1) Unlocks the file to the minimum of the argument passed to
  1562         -        **      the xUnlock() call and the current lock from SQLite's point
  1563         -        **      of view, and
  1564         -        **
  1565         -        **   2) Only unlocks the file at all if this event is the last
  1566         -        **      ASYNC_UNLOCK event on this file in the write-queue.
  1567         -        */ 
  1568         -        assert( holdingMutex==1 );
  1569         -        assert( async.pQueueFirst==p );
  1570         -        for(pIter=async.pQueueFirst->pNext; pIter; pIter=pIter->pNext){
  1571         -          if( pIter->pFileData==pData && pIter->op==ASYNC_UNLOCK ) break;
  1572         -        }
  1573         -        if( !pIter ){
  1574         -          pthread_mutex_lock(&async.lockMutex);
  1575         -          pData->lock.eAsyncLock = MIN(
  1576         -              pData->lock.eAsyncLock, MAX(pData->lock.eLock, eLock)
  1577         -          );
  1578         -          assert(pData->lock.eAsyncLock>=pData->lock.eLock);
  1579         -          rc = getFileLock(pData->pLock);
  1580         -          pthread_mutex_unlock(&async.lockMutex);
  1581         -        }
  1582         -        break;
  1583         -      }
  1584         -
  1585         -      case ASYNC_DELETE:
  1586         -        ASYNC_TRACE(("DELETE %s\n", p->zBuf));
  1587         -        rc = pVfs->xDelete(pVfs, p->zBuf, (int)p->iOffset);
  1588         -        break;
  1589         -
  1590         -      case ASYNC_OPENEXCLUSIVE: {
  1591         -        int flags = (int)p->iOffset;
  1592         -        AsyncFileData *pData = p->pFileData;
  1593         -        ASYNC_TRACE(("OPEN %s flags=%d\n", p->zBuf, (int)p->iOffset));
  1594         -        assert(pData->pBaseRead->pMethods==0 && pData->pBaseWrite->pMethods==0);
  1595         -        rc = pVfs->xOpen(pVfs, pData->zName, pData->pBaseRead, flags, 0);
  1596         -        assert( holdingMutex==0 );
  1597         -        pthread_mutex_lock(&async.queueMutex);
  1598         -        holdingMutex = 1;
  1599         -        break;
  1600         -      }
  1601         -
  1602         -      default: assert(!"Illegal value for AsyncWrite.op");
  1603         -    }
  1604         -
  1605         -    /* If we didn't hang on to the mutex during the IO op, obtain it now
  1606         -    ** so that the AsyncWrite structure can be safely removed from the 
  1607         -    ** global write-op queue.
  1608         -    */
  1609         -    if( !holdingMutex ){
  1610         -      pthread_mutex_lock(&async.queueMutex);
  1611         -      holdingMutex = 1;
  1612         -    }
  1613         -    /* ASYNC_TRACE(("UNLINK %p\n", p)); */
  1614         -    if( p==async.pQueueLast ){
  1615         -      async.pQueueLast = 0;
  1616         -    }
  1617         -    if( !doNotFree ){
  1618         -      assert_mutex_is_held(&async.queueMutex);
  1619         -      async.pQueueFirst = p->pNext;
  1620         -      sqlite3_free(p);
  1621         -    }
  1622         -    assert( holdingMutex );
  1623         -
  1624         -    /* An IO error has occurred. We cannot report the error back to the
  1625         -    ** connection that requested the I/O since the error happened 
  1626         -    ** asynchronously.  The connection has already moved on.  There 
  1627         -    ** really is nobody to report the error to.
  1628         -    **
  1629         -    ** The file for which the error occurred may have been a database or
  1630         -    ** journal file. Regardless, none of the currently queued operations
  1631         -    ** associated with the same database should now be performed. Nor should
  1632         -    ** any subsequently requested IO on either a database or journal file 
  1633         -    ** handle for the same database be accepted until the main database
  1634         -    ** file handle has been closed and reopened.
  1635         -    **
  1636         -    ** Furthermore, no further IO should be queued or performed on any file
  1637         -    ** handle associated with a database that may have been part of a 
  1638         -    ** multi-file transaction that included the database associated with 
  1639         -    ** the IO error (i.e. a database ATTACHed to the same handle at some 
  1640         -    ** point in time).
  1641         -    */
  1642         -    if( rc!=SQLITE_OK ){
  1643         -      async.ioError = rc;
  1644         -    }
  1645         -
  1646         -    if( async.ioError && !async.pQueueFirst ){
  1647         -      pthread_mutex_lock(&async.lockMutex);
  1648         -      if( 0==async.pLock ){
  1649         -        async.ioError = SQLITE_OK;
  1650         -      }
  1651         -      pthread_mutex_unlock(&async.lockMutex);
  1652         -    }
  1653         -
  1654         -    /* Drop the queue mutex before continuing to the next write operation
  1655         -    ** in order to give other threads a chance to work with the write queue.
  1656         -    */
  1657         -    if( !async.pQueueFirst || !async.ioError ){
  1658         -      pthread_mutex_unlock(&async.queueMutex);
  1659         -      holdingMutex = 0;
  1660         -      if( async.ioDelay>0 ){
  1661         -        pVfs->xSleep(pVfs, async.ioDelay);
  1662         -      }else{
  1663         -        sched_yield();
  1664         -      }
  1665         -    }
  1666         -  }
  1667         -  
  1668         -  pthread_mutex_unlock(&async.writerMutex);
  1669         -  return 0;
  1670         -}
  1671         -
  1672         -/**************************************************************************
  1673         -** The remaining code defines a Tcl interface for testing the asynchronous
  1674         -** IO implementation in this file.
  1675         -**
  1676         -** To adapt the code to a non-TCL environment, delete or comment out
  1677         -** the code that follows.
  1678         -*/
           22  +#ifdef SQLITE_ENABLE_ASYNCIO
           23  +
           24  +#include "sqlite3async.h"
           25  +#include "sqlite3.h"
           26  +#include <assert.h>
           27  +
           28  +
           29  +struct TestAsyncGlobal {
           30  +  int isInstalled;                     /* True when async VFS is installed */
           31  +} testasync_g = { 0 };
           32  +
           33  +TCL_DECLARE_MUTEX(testasync_g_writerMutex);
  1679     34   
  1680     35   /*
  1681     36   ** sqlite3async_enable ?YES/NO?
  1682     37   **
  1683     38   ** Enable or disable the asynchronous I/O backend.  This command is
  1684     39   ** not thread-safe.  Do not call it while any database connections
  1685     40   ** are open.
................................................................................
  1691     46     Tcl_Obj *CONST objv[]
  1692     47   ){
  1693     48     if( objc!=1 && objc!=2 ){
  1694     49       Tcl_WrongNumArgs(interp, 1, objv, "?YES/NO?");
  1695     50       return TCL_ERROR;
  1696     51     }
  1697     52     if( objc==1 ){
  1698         -    Tcl_SetObjResult(interp, Tcl_NewBooleanObj(async_vfs.pAppData!=0));
           53  +    Tcl_SetObjResult(interp, Tcl_NewIntObj(testasync_g.isInstalled));
  1699     54     }else{
  1700         -    int en;
  1701         -    if( Tcl_GetBooleanFromObj(interp, objv[1], &en) ) return TCL_ERROR;
  1702         -    asyncEnable(en);
           55  +    int enable;
           56  +    if( Tcl_GetBooleanFromObj(interp, objv[1], &enable) ) return TCL_ERROR;
           57  +    if( enable ){
           58  +      sqlite3async_initialize(0, 1);
           59  +    }else{
           60  +      sqlite3async_shutdown();
           61  +    }
           62  +    testasync_g.isInstalled = enable;
  1703     63     }
  1704     64     return TCL_OK;
  1705     65   }
  1706     66   
  1707     67   /*
  1708         -** sqlite3async_halt  "now"|"idle"|"never"
           68  +** sqlite3async_halt  ?"now"|"idle"|"never"?
  1709     69   **
  1710     70   ** Set the conditions at which the writer thread will halt.
  1711     71   */
  1712     72   static int testAsyncHalt(
  1713     73     void * clientData,
  1714     74     Tcl_Interp *interp,
  1715     75     int objc,
  1716     76     Tcl_Obj *CONST objv[]
  1717     77   ){
  1718         -  const char *zCond;
  1719         -  if( objc!=2 ){
  1720         -    Tcl_WrongNumArgs(interp, 1, objv, "\"now\"|\"idle\"|\"never\"");
           78  +  int eWhen;
           79  +  const char *azConstant[] = { "never", "now", "idle", 0 };
           80  +
           81  +  assert( SQLITEASYNC_HALT_NEVER==0 );
           82  +  assert( SQLITEASYNC_HALT_NOW==1 );
           83  +  assert( SQLITEASYNC_HALT_IDLE==2 );
           84  +
           85  +  if( objc!=1 && objc!=2 ){
           86  +    Tcl_WrongNumArgs(interp, 1, objv, "?OPTION?");
  1721     87       return TCL_ERROR;
  1722     88     }
  1723         -  zCond = Tcl_GetString(objv[1]);
  1724         -  if( strcmp(zCond, "now")==0 ){
  1725         -    async.writerHaltNow = 1;
  1726         -    pthread_cond_broadcast(&async.queueSignal);
  1727         -  }else if( strcmp(zCond, "idle")==0 ){
  1728         -    async.writerHaltWhenIdle = 1;
  1729         -    async.writerHaltNow = 0;
  1730         -    pthread_cond_broadcast(&async.queueSignal);
  1731         -  }else if( strcmp(zCond, "never")==0 ){
  1732         -    async.writerHaltWhenIdle = 0;
  1733         -    async.writerHaltNow = 0;
  1734         -  }else{
  1735         -    Tcl_AppendResult(interp, 
  1736         -      "should be one of: \"now\", \"idle\", or \"never\"", (char*)0);
  1737         -    return TCL_ERROR;
           89  +  if( objc==2 ){
           90  +    if( Tcl_GetIndexFromObj(interp, objv[1], azConstant, "option", 0, &eWhen) ){
           91  +      return TCL_ERROR;
           92  +    }
           93  +    sqlite3async_control(SQLITEASYNC_HALT, eWhen);
  1738     94     }
           95  +
           96  +  /* Always return the current value of the 'halt' option. */
           97  +  sqlite3async_control(SQLITEASYNC_GET_HALT, &eWhen);
           98  +  Tcl_SetObjResult(interp, Tcl_NewStringObj(azConstant[eWhen], -1));
           99  +
  1739    100     return TCL_OK;
  1740    101   }
  1741    102   
  1742    103   /*
  1743    104   ** sqlite3async_delay ?MS?
  1744    105   **
  1745    106   ** Query or set the number of milliseconds of delay in the writer
................................................................................
  1748    109   */
  1749    110   static int testAsyncDelay(
  1750    111     void * clientData,
  1751    112     Tcl_Interp *interp,
  1752    113     int objc,
  1753    114     Tcl_Obj *CONST objv[]
  1754    115   ){
          116  +  int iMs;
  1755    117     if( objc!=1 && objc!=2 ){
  1756    118       Tcl_WrongNumArgs(interp, 1, objv, "?MS?");
  1757    119       return TCL_ERROR;
  1758    120     }
  1759         -  if( objc==1 ){
  1760         -    Tcl_SetObjResult(interp, Tcl_NewIntObj(async.ioDelay));
  1761         -  }else{
  1762         -    int ioDelay;
  1763         -    if( Tcl_GetIntFromObj(interp, objv[1], &ioDelay) ) return TCL_ERROR;
  1764         -    async.ioDelay = ioDelay;
          121  +  if( objc==2 ){
          122  +    if( Tcl_GetIntFromObj(interp, objv[1], &iMs) ){
          123  +      return TCL_ERROR;
          124  +    }
          125  +    sqlite3async_control(SQLITEASYNC_DELAY, iMs);
  1765    126     }
          127  +
          128  +  /* Always return the current value of the 'delay' option. */
          129  +  sqlite3async_control(SQLITEASYNC_GET_DELAY, &iMs);
          130  +  Tcl_SetObjResult(interp, Tcl_NewIntObj(iMs));
  1766    131     return TCL_OK;
  1767    132   }
          133  +
          134  +static Tcl_ThreadCreateType tclWriterThread(ClientData pIsStarted){
          135  +  Tcl_MutexLock(&testasync_g_writerMutex);
          136  +  *((int *)pIsStarted) = 1;
          137  +  sqlite3async_run();
          138  +  Tcl_MutexUnlock(&testasync_g_writerMutex);
          139  +  TCL_THREAD_CREATE_RETURN;
          140  +}
  1768    141   
  1769    142   /*
  1770    143   ** sqlite3async_start
  1771    144   **
  1772    145   ** Start a new writer thread.
  1773    146   */
  1774    147   static int testAsyncStart(
  1775    148     void * clientData,
  1776    149     Tcl_Interp *interp,
  1777    150     int objc,
  1778    151     Tcl_Obj *CONST objv[]
  1779    152   ){
  1780         -  pthread_t x;
  1781         -  int rc;
  1782    153     volatile int isStarted = 0;
  1783         -  rc = pthread_create(&x, 0, asyncWriterThread, (void *)&isStarted);
  1784         -  if( rc ){
  1785         -    Tcl_AppendResult(interp, "failed to create the thread", 0);
          154  +  ClientData threadData = (ClientData)&isStarted;
          155  +
          156  +  Tcl_ThreadId x;
          157  +  const int nStack = TCL_THREAD_STACK_DEFAULT;
          158  +  const int flags = TCL_THREAD_NOFLAGS;
          159  +  int rc;
          160  +
          161  +  rc = Tcl_CreateThread(&x, tclWriterThread, threadData, nStack, flags);
          162  +  if( rc!=TCL_OK ){
  1786    163       return TCL_ERROR;
  1787    164     }
  1788         -  pthread_detach(x);
  1789    165     while( isStarted==0 ){
          166  +#if 0
  1790    167       sched_yield();
          168  +#endif
  1791    169     }
  1792    170     return TCL_OK;
  1793    171   }
  1794    172   
  1795    173   /*
  1796    174   ** sqlite3async_wait
  1797    175   **
................................................................................
  1802    180   */
  1803    181   static int testAsyncWait(
  1804    182     void * clientData,
  1805    183     Tcl_Interp *interp,
  1806    184     int objc,
  1807    185     Tcl_Obj *CONST objv[]
  1808    186   ){
  1809         -  int cnt = 10;
  1810         -  if( async.writerHaltNow==0 && async.writerHaltWhenIdle==0 ){
          187  +  int eCond;
          188  +  if( objc!=1 ){
          189  +    Tcl_WrongNumArgs(interp, 1, objv, "");
          190  +    return TCL_ERROR;
          191  +  }
          192  +
          193  +  sqlite3async_control(SQLITEASYNC_GET_HALT, &eCond);
          194  +  if( eCond==SQLITEASYNC_HALT_NEVER ){
  1811    195       Tcl_AppendResult(interp, "would block forever", (char*)0);
  1812    196       return TCL_ERROR;
  1813    197     }
  1814    198   
  1815         -  while( cnt-- && !pthread_mutex_trylock(&async.writerMutex) ){
  1816         -    pthread_mutex_unlock(&async.writerMutex);
  1817         -    sched_yield();
  1818         -  }
  1819         -  if( cnt>=0 ){
  1820         -    ASYNC_TRACE(("WAIT\n"));
  1821         -    pthread_mutex_lock(&async.queueMutex);
  1822         -    pthread_cond_broadcast(&async.queueSignal);
  1823         -    pthread_mutex_unlock(&async.queueMutex);
  1824         -    pthread_mutex_lock(&async.writerMutex);
  1825         -    pthread_mutex_unlock(&async.writerMutex);
  1826         -  }else{
  1827         -    ASYNC_TRACE(("NO-WAIT\n"));
  1828         -  }
          199  +  Tcl_MutexLock(&testasync_g_writerMutex);
          200  +  Tcl_MutexUnlock(&testasync_g_writerMutex);
  1829    201     return TCL_OK;
  1830    202   }
  1831    203   
  1832         -
  1833         -#endif  /* SQLITE_OS_UNIX and SQLITE_THREADSAFE */
          204  +#endif  /* SQLITE_ENABLE_ASYNCIO */
  1834    205   
  1835    206   /*
  1836    207   ** This routine registers the custom TCL commands defined in this
  1837    208   ** module.  This should be the only procedure visible from outside
  1838    209   ** of this module.
  1839    210   */
  1840    211   int Sqlitetestasync_Init(Tcl_Interp *interp){
  1841         -#if SQLITE_OS_UNIX && SQLITE_THREADSAFE
          212  +#if SQLITE_ENABLE_ASYNCIO
  1842    213     Tcl_CreateObjCommand(interp,"sqlite3async_enable",testAsyncEnable,0,0);
  1843    214     Tcl_CreateObjCommand(interp,"sqlite3async_halt",testAsyncHalt,0,0);
  1844    215     Tcl_CreateObjCommand(interp,"sqlite3async_delay",testAsyncDelay,0,0);
  1845    216     Tcl_CreateObjCommand(interp,"sqlite3async_start",testAsyncStart,0,0);
  1846    217     Tcl_CreateObjCommand(interp,"sqlite3async_wait",testAsyncWait,0,0);
  1847         -  Tcl_LinkVar(interp, "sqlite3async_trace",
  1848         -      (char*)&sqlite3async_trace, TCL_LINK_INT);
  1849         -#endif  /* SQLITE_OS_UNIX and SQLITE_THREADSAFE */
          218  +#endif  /* SQLITE_ENABLE_ASYNCIO */
  1850    219     return TCL_OK;
  1851    220   }
          221  +

Changes to test/insert3.test.

     7      7   #    May you find forgiveness for yourself and forgive others.
     8      8   #    May you share freely, never taking more than you give.
     9      9   #
    10     10   #***********************************************************************
    11     11   # This file implements regression tests for SQLite library.  The
    12     12   # focus of this file is testing corner cases of the INSERT statement.
    13     13   #
    14         -# $Id: insert3.test,v 1.8 2008/12/23 10:37:47 danielk1977 Exp $
           14  +# $Id: insert3.test,v 1.9 2009/04/23 14:58:40 danielk1977 Exp $
    15     15   
    16     16   set testdir [file dirname $argv0]
    17     17   source $testdir/tester.tcl
    18     18   
    19     19   # All the tests in this file require trigger support
    20     20   #
    21     21   ifcapable {trigger} {
................................................................................
   162    162       execsql {
   163    163         CREATE TABLE t6(x,y DEFAULT 4.3, z DEFAULT x'6869');
   164    164         INSERT INTO t6 DEFAULT VALUES;
   165    165         SELECT * FROM t6;
   166    166       }
   167    167     } {{} 4.3 hi}
   168    168   }
   169         -db close
   170    169   
   171         -file delete -force test.db
          170  +foreach tab [db eval {SELECT name FROM sqlite_master WHERE type = 'table'}] {
          171  +  db eval "DROP TABLE $tab"
          172  +}
          173  +db close
   172    174   sqlite3 db test.db
   173    175   
   174    176   #-------------------------------------------------------------------------
   175    177   # While developing tests for a different feature (savepoint) the following
   176    178   # sequence was found to cause an assert() in btree.c to fail. These
   177    179   # tests are included to ensure that that bug is fixed.
   178    180   #

Changes to test/mutex1.test.

     5      5   #
     6      6   #    May you do good and not evil.
     7      7   #    May you find forgiveness for yourself and forgive others.
     8      8   #    May you share freely, never taking more than you give.
     9      9   #
    10     10   #***********************************************************************
    11     11   #
    12         -# $Id: mutex1.test,v 1.19 2009/03/24 15:08:10 drh Exp $
           12  +# $Id: mutex1.test,v 1.20 2009/04/23 14:58:40 danielk1977 Exp $
    13     13   
    14     14   set testdir [file dirname $argv0]
    15     15   source $testdir/tester.tcl
    16     16   
    17     17   ifcapable !mutex {
    18     18     finish_test
    19     19     return
................................................................................
    93     93   # Tests mutex1-2.* test the three thread-safety related modes that
    94     94   # can be selected using sqlite3_config:
    95     95   #
    96     96   #   * Serialized mode,
    97     97   #   * Multi-threaded mode,
    98     98   #   * Single-threaded mode.
    99     99   #
   100         -set enable_shared_cache [sqlite3_enable_shared_cache 1]
   101         -ifcapable threadsafe {
          100  +ifcapable threadsafe&&shared_cache {
          101  +  set enable_shared_cache [sqlite3_enable_shared_cache 1]
   102    102     foreach {mode mutexes} {
   103    103       singlethread {}
   104    104       multithread  {fast static_lru static_master static_mem static_open static_prng }
   105    105       serialized  {fast recursive static_lru static_master static_mem static_open static_prng}
   106    106     } {
   107    107   
   108    108       do_test mutex1.2.$mode.1 {
................................................................................
   129    129           if {$key ne "total" && $value > 0} {
   130    130             lappend res $key
   131    131           }
   132    132         }
   133    133         lsort $res
   134    134       } [lsort $mutexes]
   135    135     }
   136         -}
   137         -sqlite3_enable_shared_cache $enable_shared_cache
          136  +  sqlite3_enable_shared_cache $enable_shared_cache
   138    137   
   139         -# Open and use a connection in "nomutex" mode. Test that no recursive
   140         -# mutexes are obtained.
   141         -ifcapable threadsafe {
          138  +  # Open and use a connection in "nomutex" mode. Test that no recursive
          139  +  # mutexes are obtained.
   142    140     do_test mutex1.3.1 {
   143    141       catch {db close}
   144    142       clear_mutex_counters
   145    143       sqlite3 db test.db -nomutex 1
   146    144       execsql { SELECT * FROM abc }
   147    145     } {1 2 3 1 2 3 1 2 3}
   148    146     do_test mutex1.3.2 {

Changes to test/select1.test.

     7      7   #    May you find forgiveness for yourself and forgive others.
     8      8   #    May you share freely, never taking more than you give.
     9      9   #
    10     10   #***********************************************************************
    11     11   # This file implements regression tests for SQLite library.  The
    12     12   # focus of this file is testing the SELECT statement.
    13     13   #
    14         -# $Id: select1.test,v 1.67 2009/04/10 15:38:43 drh Exp $
           14  +# $Id: select1.test,v 1.68 2009/04/23 14:58:40 danielk1977 Exp $
    15     15   
    16     16   set testdir [file dirname $argv0]
    17     17   source $testdir/tester.tcl
    18     18   
    19     19   # Try to select on a non-existant table.
    20     20   #
    21     21   do_test select1-1.1 {
................................................................................
  1018   1018         SELECT count(
  1019   1019           (SELECT a FROM abc WHERE a = NULL AND b >= upper.c) 
  1020   1020         ) FROM abc AS upper;
  1021   1021       }
  1022   1022     } {0}
  1023   1023   }
  1024   1024   
         1025  +foreach tab [db eval {SELECT name FROM sqlite_master WHERE type = 'table'}] {
         1026  +  db eval "DROP TABLE $tab"
         1027  +}
  1025   1028   db close
  1026         -file delete -force test.db
  1027   1029   sqlite3 db test.db
         1030  +
  1028   1031   do_test select1-14.1 {
  1029   1032     execsql { 
  1030   1033       SELECT * FROM sqlite_master WHERE rowid>10; 
  1031   1034       SELECT * FROM sqlite_master WHERE rowid=10;
  1032   1035       SELECT * FROM sqlite_master WHERE rowid<10;
  1033   1036       SELECT * FROM sqlite_master WHERE rowid<=10;
  1034   1037       SELECT * FROM sqlite_master WHERE rowid>=10;