/ Check-in [fde4c597]
Login
SQLite training in Houston TX on 2019-11-05 (details)
Part of the 2019 Tcl Conference

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add comments to clarify the purpose of the pager "error state".
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | experimental
Files: files | file ages | folders
SHA1: fde4c59782e98366792e1552fd122e7d80a277ff
User & Date: dan 2010-08-03 15:57:10
Context
2010-08-03
18:18
Set the Pager.eState variable to PAGER_ERROR whenever the pager enters the error state. check-in: 4d384761 user: dan tags: experimental
15:57
Add comments to clarify the purpose of the pager "error state". check-in: fde4c597 user: dan tags: experimental
12:48
Add state diagram to comments in experimental version of pager.c. check-in: 16dcf5a6 user: dan tags: experimental
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to src/pager.c.

   136    136   **    unknown. The database may not be read or written.
   137    137   **
   138    138   **    * No read or write transaction is active.
   139    139   **    * Any lock, or no lock at all, may be held on the database file.
   140    140   **    * The dbSize, dbOrigSize and dbFileSize variables may not be trusted.
   141    141   **
   142    142   **  READER:
          143  +**
   143    144   **    In this state all the requirements for reading the database in 
   144    145   **    rollback (non-WAL) mode are met. Unless the pager is (or recently
   145    146   **    was) in exclusive-locking mode, a user-level read transaction is 
   146    147   **    open. The database size is known in this state.
   147    148   ** 
   148    149   **    * A read transaction may be active.
   149    150   **    * A SHARED or greater lock is held on the database file.
   150    151   **    * The dbSize variable may be trusted (even if a user-level read 
   151    152   **      transaction is not active). The dbOrigSize and dbFileSize variables
   152    153   **      may not be trusted at this point.
   153    154   **
   154    155   **  WRITER_INITIAL:
          156  +**
   155    157   **    * A write transaction is active.
   156    158   **    * A RESERVED or greater lock is held on the database file.
   157    159   **    * The dbSize, dbOrigSize and dbFileSize variables are all valid.
   158    160   **    * The contents of the pager cache have not been modified.
          161  +**    * The journal file may or may not be open.
          162  +**    * Nothing (not even the first header) has been written to the journal.
   159    163   **
   160    164   **  WRITER_CACHEMOD:
          165  +**
   161    166   **    * A write transaction is active.
   162    167   **    * A RESERVED or greater lock is held on the database file.
   163    168   **    * The journal file is open and the first header has been written 
   164    169   **      to it, but the header has not been synced to disk.
   165    170   **    * The contents of the page cache have been modified.
   166    171   **
   167    172   **  WRITER_DBMOD:
          173  +**
   168    174   **    * A write transaction is active.
   169    175   **    * An EXCLUSIVE or greater lock is held on the database file.
   170    176   **    * The journal file is open and the first header has been written 
   171    177   **      and synced to disk.
   172    178   **    * The contents of the page cache have been modified (and possibly
   173    179   **      written to disk).
   174    180   **
   175    181   **  WRITER_FINISHED:
          182  +**
   176    183   **    * A write transaction is active.
   177    184   **    * An EXCLUSIVE or greater lock is held on the database file.
   178    185   **    * All writing and syncing of journal and database data has finished.
   179    186   **      If no error occured, all that remains is to finalize the journal to
   180    187   **      commit the transaction. If an error did occur, the caller will need
   181    188   **      to rollback the transaction. 
          189  +**
          190  +**  ERROR:
          191  +**
          192  +**    The ERROR state is entered when an IO, OOM or disk-full error 
          193  +**    occurs at a point in the code that makes it difficult to be sure
          194  +**    that the in-memory pager state (cache contents, db size etc.) are
          195  +**    consistent with the contents of the file-system.
          196  +**
          197  +**    For example, if an IO error occurs while performing a rollback, 
          198  +**    the contents of the page-cache may be left in an inconsistent state.
          199  +**    At this point it would be dangerous to change back to READER state
          200  +**    (as usually happens after a rollback). Any subsequent readers might
          201  +**    report database corruption (due to the inconsistent cache), and if
          202  +**    they upgrade to writers, they may inadvertently corrupt the database
          203  +**    file. To avoid this hazard, the pager switches into the ERROR state
          204  +**    instead of READER following such an error.
          205  +**
          206  +**    Once it has entered the ERROR state, any attempt to use the pager
          207  +**    to read or write data returns an error. Eventually, once all 
          208  +**    outstanding transactions have been abandoned, the pager is able to
          209  +**    transition back to NONE state, discarding the contents of the 
          210  +**    page-cache and any other in-memory state at the same time. Everything
          211  +**    is reloaded from disk (and, if necessary, hot-journal rollback peformed)
          212  +**    when a read-transaction is next opened on the pager (transitioning
          213  +**    the pager into READER state). At that point the system has recovered 
          214  +**    from the error.
          215  +**
          216  +**    Specifically, the pager jumps into the ERROR state if:
          217  +**
          218  +**      1. An error occurs while attempting a rollback. This happens in
          219  +**         function sqlite3PagerRollback().
          220  +**
          221  +**      2. An error occurs while attempting to finalize a journal file
          222  +**         following a commit in function sqlite3PagerCommitPhaseTwo().
          223  +**
          224  +**      3. An error occurs while attempting to write to the journal or
          225  +**         database file in function pagerStress() in order to free up
          226  +**         memory.
          227  +**
          228  +**    In other cases, the error is returned to the b-tree layer. The b-tree
          229  +**    layer then attempts a rollback operation. If the error condition 
          230  +**    persists, the pager enters the ERROR state via condition (1) above.
          231  +**
          232  +**    Condition (3) is necessary because it can be triggered by a read-only
          233  +**    statement executed within a transaction. In this case, if the error
          234  +**    code were simply returned to the user, the b-tree layer would not
          235  +**    automatically attempt a rollback, as it assumes that an error in a
          236  +**    read-only statement cannot leave the pager in an internally inconsistent 
          237  +**    state.
          238  +**    
   182    239   **
   183    240   ** State diagram:
   184    241   **
   185         -**                            NONE <-------------+
   186         -**                              |                |
   187         -**                              V                |
   188         -**               +---------> READER------------->|
          242  +**                            NONE <------+------+
          243  +**                              |         |      |
          244  +**                              V         |      |
          245  +**               +---------> READER-------+      |
   189    246   **               |              |                |
   190    247   **               |              V                |
   191    248   **               |<-------WRITER_INITIAL-----> ERROR
   192    249   **               |              |                ^  
   193    250   **               |              V                |
   194    251   **               |<------WRITER_CACHEMOD-------->|
   195    252   **               |              |                |
................................................................................
  1615   1672           rc = sqlite3OsTruncate(pPager->jfd, 0);
  1616   1673         }
  1617   1674         pPager->journalOff = 0;
  1618   1675       }else if( pPager->journalMode==PAGER_JOURNALMODE_PERSIST
  1619   1676         || (pPager->exclusiveMode && pPager->journalMode!=PAGER_JOURNALMODE_WAL)
  1620   1677       ){
  1621   1678         rc = zeroJournalHdr(pPager, hasMaster);
  1622         -      pager_error(pPager, rc);
  1623   1679         pPager->journalOff = 0;
  1624   1680       }else{
  1625   1681         /* This branch may be executed with Pager.journalMode==MEMORY if
  1626   1682         ** a hot-journal was just rolled back. In this case the journal
  1627   1683         ** file should be closed and deleted. If this connection writes to
  1628   1684         ** the database file, it will do so using an in-memory journal. 
  1629   1685         */
................................................................................
  4323   4379       assert( sqlite3PcacheRefCount(pPager->pPCache)==0 );
  4324   4380       assert( pPager->noReadlock==0 || pPager->readOnly );
  4325   4381   
  4326   4382       if( pPager->noReadlock==0 ){
  4327   4383         rc = pager_wait_on_lock(pPager, SHARED_LOCK);
  4328   4384         if( rc!=SQLITE_OK ){
  4329   4385           assert( pPager->eLock==PAGER_UNLOCK );
  4330         -        return pager_error(pPager, rc);
         4386  +        goto failed;
  4331   4387         }
  4332   4388       }
  4333   4389   
  4334   4390       /* If a journal file exists, and there is no RESERVED lock on the
  4335   4391       ** database file, then it either needs to be played back or deleted.
  4336   4392       */
  4337   4393       if( !isErrorReset ){
................................................................................
  4354   4410         ** on the database file.
  4355   4411         **
  4356   4412         ** Unless the pager is in locking_mode=exclusive mode, the lock is
  4357   4413         ** downgraded to SHARED_LOCK before this function returns.
  4358   4414         */
  4359   4415         rc = pagerLock(pPager, EXCLUSIVE_LOCK);
  4360   4416         if( rc!=SQLITE_OK ){
  4361         -        rc = pager_error(pPager, rc);
  4362   4417           goto failed;
  4363   4418         }
  4364   4419    
  4365   4420         /* If it is not already open and the file exists on disk, open the 
  4366   4421         ** journal for read/write access. Write access is required because 
  4367   4422         ** in exclusive-access mode the file descriptor will be kept open 
  4368   4423         ** and possibly used for a transaction later on. Also, write-access 
................................................................................
  4407   4462         /* Playback and delete the journal.  Drop the database write
  4408   4463         ** lock and reacquire the read lock. Purge the cache before
  4409   4464         ** playing back the hot-journal so that we don't end up with
  4410   4465         ** an inconsistent cache.  Sync the hot journal before playing
  4411   4466         ** it back since the process that crashed and left the hot journal
  4412   4467         ** probably did not sync it and we are required to always sync
  4413   4468         ** the journal before playing it back.
         4469  +      **
         4470  +      ** Even if an error occurs while syncing or rolling back the 
         4471  +      ** hot-journal, there is no need to enter the ERROR state here, as
         4472  +      ** the pager never left state NONE anyhow.
  4414   4473         */
  4415   4474         if( isOpen(pPager->jfd) ){
  4416   4475           rc = pagerSyncHotJournal(pPager);
  4417   4476           if( rc==SQLITE_OK ){
  4418   4477             pPager->eState = PAGER_WRITER_FINISHED;
  4419   4478             rc = pager_playback(pPager, 1);
  4420   4479             pPager->eState = PAGER_NONE;
  4421   4480           }
  4422   4481           if( rc!=SQLITE_OK ){
  4423         -          rc = pager_error(pPager, rc);
  4424   4482             goto failed;
  4425   4483           }
  4426   4484         }else{
  4427   4485           osUnlock(pPager, SHARED_LOCK);
  4428   4486         }
  4429   4487   
  4430   4488         assert( pPager->eState==PAGER_NONE );
................................................................................
  4484   4542     if( pPager->eState==PAGER_NONE && rc==SQLITE_OK ){
  4485   4543       pPager->eState = PAGER_NONE;
  4486   4544       rc = pagerPagecount(pPager, &pPager->dbSize);
  4487   4545     }
  4488   4546   
  4489   4547    failed:
  4490   4548     if( rc!=SQLITE_OK ){
  4491         -    /* pager_unlock() is a no-op for exclusive mode and in-memory databases. */
  4492   4549       pager_unlock(pPager);
         4550  +    assert( (pPager->eState==PAGER_NONE)
         4551  +        ||  (pPager->exclusiveMode && pagerUseWal(pPager))
         4552  +    );
  4493   4553     }else{
  4494   4554       pPager->eState = PAGER_READER;
  4495   4555     }
  4496   4556     return rc;
  4497   4557   }
  4498   4558   
  4499   4559   /*
................................................................................
  5540   5600     int rc = SQLITE_OK;                  /* Return code */
  5541   5601   
  5542   5602     /* This routine should not be called if a prior error has occurred.
  5543   5603     ** But if (due to a coding error elsewhere in the system) it does get
  5544   5604     ** called, just return the same error code without doing anything. */
  5545   5605     if( NEVER(pPager->errCode) ) return pPager->errCode;
  5546   5606   
  5547         -  /* This function should not be called if the pager is not in at least
  5548         -  ** PAGER_RESERVED state. **FIXME**: Make it so that this test always
  5549         -  ** fails - make it so that we never reach this point if we do not hold
  5550         -  ** all necessary locks.
  5551         -  */
  5552   5607     assert( pPager->eState==PAGER_WRITER_INITIAL
  5553   5608          || pPager->eState==PAGER_WRITER_FINISHED
  5554   5609          || (pagerUseWal(pPager) && pPager->eState==PAGER_WRITER_CACHEMOD)
  5555   5610     );
  5556   5611     assert( assert_pager_state(pPager) );
  5557   5612   
  5558         -#if 0
  5559         -  if( NEVER(pPager->state<PAGER_RESERVED) ) return SQLITE_ERROR;
  5560         -#endif
  5561         -
  5562   5613     /* An optimization. If the database was not actually modified during
  5563   5614     ** this transaction, the pager is running in exclusive-mode and is
  5564   5615     ** using persistent journals, then this function is a no-op.
  5565   5616     **
  5566   5617     ** The start of the journal file currently contains a single journal 
  5567   5618     ** header with the nRec field set to 0. If such a journal is used as
  5568   5619     ** a hot-journal during hot-journal rollback, 0 changes will be made
................................................................................
  5633   5684   
  5634   5685     /* PagerRollback() is a no-op if called in READER or NONE state. */
  5635   5686     assert( assert_pager_state(pPager) );
  5636   5687     if( pPager->eState<=PAGER_READER ) return SQLITE_OK;
  5637   5688   
  5638   5689     if( pagerUseWal(pPager) ){
  5639   5690       int rc2;
  5640         -
  5641   5691       rc = sqlite3PagerSavepoint(pPager, SAVEPOINT_ROLLBACK, -1);
  5642   5692       rc2 = pager_end_transaction(pPager, pPager->setMaster);
  5643   5693       if( rc==SQLITE_OK ) rc = rc2;
  5644         -    rc = pager_error(pPager, rc);
  5645   5694     }else if( pPager->eState==PAGER_WRITER_INITIAL ){
  5646         -    rc = pager_end_transaction(pPager, pPager->setMaster);
         5695  +    rc = pager_end_transaction(pPager, 0);
  5647   5696     }else if( pPager->errCode && pPager->errCode!=SQLITE_FULL ){
  5648   5697       if( pPager->eState>=PAGER_WRITER_DBMOD ){
  5649   5698         pager_playback(pPager, 0);
  5650   5699       }
  5651   5700       rc = pPager->errCode;
  5652   5701     }else{
  5653   5702       if( pPager->eState==PAGER_WRITER_CACHEMOD ){
................................................................................
  5656   5705         rc2 = pager_end_transaction(pPager, pPager->setMaster);
  5657   5706         if( rc==SQLITE_OK ){
  5658   5707           rc = rc2;
  5659   5708         }
  5660   5709       }else{
  5661   5710         rc = pager_playback(pPager, 0);
  5662   5711       }
         5712  +  }
  5663   5713   
  5664         -    /* If an error occurs during a ROLLBACK, we can no longer trust the pager
  5665         -    ** cache. So call pager_error() on the way out to make any error 
  5666         -    ** persistent.
  5667         -    */
  5668         -    rc = pager_error(pPager, rc);
  5669         -  }
  5670         -  return rc;
         5714  +  /* If an error occurs during a ROLLBACK, we can no longer trust the pager
         5715  +  ** cache. So call pager_error() on the way out to make any error persistent.
         5716  +  */
         5717  +  return pager_error(pPager, rc);
  5671   5718   }
  5672   5719   
  5673   5720   /*
  5674   5721   ** Return TRUE if the database file is opened read-only.  Return FALSE
  5675   5722   ** if the database is (in theory) writable.
  5676   5723   */
  5677   5724   u8 sqlite3PagerIsreadonly(Pager *pPager){