SQLite

Check-in [fde4c59782]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add comments to clarify the purpose of the pager "error state".
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | experimental
Files: files | file ages | folders
SHA1: fde4c59782e98366792e1552fd122e7d80a277ff
User & Date: dan 2010-08-03 15:57:10.000
Context
2010-08-03
18:18
Set the Pager.eState variable to PAGER_ERROR whenever the pager enters the error state. (check-in: 4d384761d2 user: dan tags: experimental)
15:57
Add comments to clarify the purpose of the pager "error state". (check-in: fde4c59782 user: dan tags: experimental)
12:48
Add state diagram to comments in experimental version of pager.c. (check-in: 16dcf5a6d3 user: dan tags: experimental)
Changes
Unified Diff Ignore Whitespace Patch
Changes to src/pager.c.
136
137
138
139
140
141
142

143
144
145
146
147
148
149
150
151
152
153
154

155
156
157
158


159
160

161
162
163
164
165
166
167

168
169
170
171
172
173
174
175

176
177
178
179
180
181
182


















































183
184
185
186
187
188
189
190
191
192
193
194
195
**    unknown. The database may not be read or written.
**
**    * No read or write transaction is active.
**    * Any lock, or no lock at all, may be held on the database file.
**    * The dbSize, dbOrigSize and dbFileSize variables may not be trusted.
**
**  READER:

**    In this state all the requirements for reading the database in 
**    rollback (non-WAL) mode are met. Unless the pager is (or recently
**    was) in exclusive-locking mode, a user-level read transaction is 
**    open. The database size is known in this state.
** 
**    * A read transaction may be active.
**    * A SHARED or greater lock is held on the database file.
**    * The dbSize variable may be trusted (even if a user-level read 
**      transaction is not active). The dbOrigSize and dbFileSize variables
**      may not be trusted at this point.
**
**  WRITER_INITIAL:

**    * A write transaction is active.
**    * A RESERVED or greater lock is held on the database file.
**    * The dbSize, dbOrigSize and dbFileSize variables are all valid.
**    * The contents of the pager cache have not been modified.


**
**  WRITER_CACHEMOD:

**    * A write transaction is active.
**    * A RESERVED or greater lock is held on the database file.
**    * The journal file is open and the first header has been written 
**      to it, but the header has not been synced to disk.
**    * The contents of the page cache have been modified.
**
**  WRITER_DBMOD:

**    * A write transaction is active.
**    * An EXCLUSIVE or greater lock is held on the database file.
**    * The journal file is open and the first header has been written 
**      and synced to disk.
**    * The contents of the page cache have been modified (and possibly
**      written to disk).
**
**  WRITER_FINISHED:

**    * A write transaction is active.
**    * An EXCLUSIVE or greater lock is held on the database file.
**    * All writing and syncing of journal and database data has finished.
**      If no error occured, all that remains is to finalize the journal to
**      commit the transaction. If an error did occur, the caller will need
**      to rollback the transaction. 
**


















































** State diagram:
**
**                            NONE <-------------+
**                              |                |
**                              V                |
**               +---------> READER------------->|
**               |              |                |
**               |              V                |
**               |<-------WRITER_INITIAL-----> ERROR
**               |              |                ^  
**               |              V                |
**               |<------WRITER_CACHEMOD-------->|
**               |              |                |







>












>




>
>


>







>








>







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


|
|
|
|







136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
**    unknown. The database may not be read or written.
**
**    * No read or write transaction is active.
**    * Any lock, or no lock at all, may be held on the database file.
**    * The dbSize, dbOrigSize and dbFileSize variables may not be trusted.
**
**  READER:
**
**    In this state all the requirements for reading the database in 
**    rollback (non-WAL) mode are met. Unless the pager is (or recently
**    was) in exclusive-locking mode, a user-level read transaction is 
**    open. The database size is known in this state.
** 
**    * A read transaction may be active.
**    * A SHARED or greater lock is held on the database file.
**    * The dbSize variable may be trusted (even if a user-level read 
**      transaction is not active). The dbOrigSize and dbFileSize variables
**      may not be trusted at this point.
**
**  WRITER_INITIAL:
**
**    * A write transaction is active.
**    * A RESERVED or greater lock is held on the database file.
**    * The dbSize, dbOrigSize and dbFileSize variables are all valid.
**    * The contents of the pager cache have not been modified.
**    * The journal file may or may not be open.
**    * Nothing (not even the first header) has been written to the journal.
**
**  WRITER_CACHEMOD:
**
**    * A write transaction is active.
**    * A RESERVED or greater lock is held on the database file.
**    * The journal file is open and the first header has been written 
**      to it, but the header has not been synced to disk.
**    * The contents of the page cache have been modified.
**
**  WRITER_DBMOD:
**
**    * A write transaction is active.
**    * An EXCLUSIVE or greater lock is held on the database file.
**    * The journal file is open and the first header has been written 
**      and synced to disk.
**    * The contents of the page cache have been modified (and possibly
**      written to disk).
**
**  WRITER_FINISHED:
**
**    * A write transaction is active.
**    * An EXCLUSIVE or greater lock is held on the database file.
**    * All writing and syncing of journal and database data has finished.
**      If no error occured, all that remains is to finalize the journal to
**      commit the transaction. If an error did occur, the caller will need
**      to rollback the transaction. 
**
**  ERROR:
**
**    The ERROR state is entered when an IO, OOM or disk-full error 
**    occurs at a point in the code that makes it difficult to be sure
**    that the in-memory pager state (cache contents, db size etc.) are
**    consistent with the contents of the file-system.
**
**    For example, if an IO error occurs while performing a rollback, 
**    the contents of the page-cache may be left in an inconsistent state.
**    At this point it would be dangerous to change back to READER state
**    (as usually happens after a rollback). Any subsequent readers might
**    report database corruption (due to the inconsistent cache), and if
**    they upgrade to writers, they may inadvertently corrupt the database
**    file. To avoid this hazard, the pager switches into the ERROR state
**    instead of READER following such an error.
**
**    Once it has entered the ERROR state, any attempt to use the pager
**    to read or write data returns an error. Eventually, once all 
**    outstanding transactions have been abandoned, the pager is able to
**    transition back to NONE state, discarding the contents of the 
**    page-cache and any other in-memory state at the same time. Everything
**    is reloaded from disk (and, if necessary, hot-journal rollback peformed)
**    when a read-transaction is next opened on the pager (transitioning
**    the pager into READER state). At that point the system has recovered 
**    from the error.
**
**    Specifically, the pager jumps into the ERROR state if:
**
**      1. An error occurs while attempting a rollback. This happens in
**         function sqlite3PagerRollback().
**
**      2. An error occurs while attempting to finalize a journal file
**         following a commit in function sqlite3PagerCommitPhaseTwo().
**
**      3. An error occurs while attempting to write to the journal or
**         database file in function pagerStress() in order to free up
**         memory.
**
**    In other cases, the error is returned to the b-tree layer. The b-tree
**    layer then attempts a rollback operation. If the error condition 
**    persists, the pager enters the ERROR state via condition (1) above.
**
**    Condition (3) is necessary because it can be triggered by a read-only
**    statement executed within a transaction. In this case, if the error
**    code were simply returned to the user, the b-tree layer would not
**    automatically attempt a rollback, as it assumes that an error in a
**    read-only statement cannot leave the pager in an internally inconsistent 
**    state.
**    
**
** State diagram:
**
**                            NONE <------+------+
**                              |         |      |
**                              V         |      |
**               +---------> READER-------+      |
**               |              |                |
**               |              V                |
**               |<-------WRITER_INITIAL-----> ERROR
**               |              |                ^  
**               |              V                |
**               |<------WRITER_CACHEMOD-------->|
**               |              |                |
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
        rc = sqlite3OsTruncate(pPager->jfd, 0);
      }
      pPager->journalOff = 0;
    }else if( pPager->journalMode==PAGER_JOURNALMODE_PERSIST
      || (pPager->exclusiveMode && pPager->journalMode!=PAGER_JOURNALMODE_WAL)
    ){
      rc = zeroJournalHdr(pPager, hasMaster);
      pager_error(pPager, rc);
      pPager->journalOff = 0;
    }else{
      /* This branch may be executed with Pager.journalMode==MEMORY if
      ** a hot-journal was just rolled back. In this case the journal
      ** file should be closed and deleted. If this connection writes to
      ** the database file, it will do so using an in-memory journal. 
      */







<







1672
1673
1674
1675
1676
1677
1678

1679
1680
1681
1682
1683
1684
1685
        rc = sqlite3OsTruncate(pPager->jfd, 0);
      }
      pPager->journalOff = 0;
    }else if( pPager->journalMode==PAGER_JOURNALMODE_PERSIST
      || (pPager->exclusiveMode && pPager->journalMode!=PAGER_JOURNALMODE_WAL)
    ){
      rc = zeroJournalHdr(pPager, hasMaster);

      pPager->journalOff = 0;
    }else{
      /* This branch may be executed with Pager.journalMode==MEMORY if
      ** a hot-journal was just rolled back. In this case the journal
      ** file should be closed and deleted. If this connection writes to
      ** the database file, it will do so using an in-memory journal. 
      */
4323
4324
4325
4326
4327
4328
4329
4330
4331
4332
4333
4334
4335
4336
4337
    assert( sqlite3PcacheRefCount(pPager->pPCache)==0 );
    assert( pPager->noReadlock==0 || pPager->readOnly );

    if( pPager->noReadlock==0 ){
      rc = pager_wait_on_lock(pPager, SHARED_LOCK);
      if( rc!=SQLITE_OK ){
        assert( pPager->eLock==PAGER_UNLOCK );
        return pager_error(pPager, rc);
      }
    }

    /* If a journal file exists, and there is no RESERVED lock on the
    ** database file, then it either needs to be played back or deleted.
    */
    if( !isErrorReset ){







|







4379
4380
4381
4382
4383
4384
4385
4386
4387
4388
4389
4390
4391
4392
4393
    assert( sqlite3PcacheRefCount(pPager->pPCache)==0 );
    assert( pPager->noReadlock==0 || pPager->readOnly );

    if( pPager->noReadlock==0 ){
      rc = pager_wait_on_lock(pPager, SHARED_LOCK);
      if( rc!=SQLITE_OK ){
        assert( pPager->eLock==PAGER_UNLOCK );
        goto failed;
      }
    }

    /* If a journal file exists, and there is no RESERVED lock on the
    ** database file, then it either needs to be played back or deleted.
    */
    if( !isErrorReset ){
4354
4355
4356
4357
4358
4359
4360
4361
4362
4363
4364
4365
4366
4367
4368
      ** on the database file.
      **
      ** Unless the pager is in locking_mode=exclusive mode, the lock is
      ** downgraded to SHARED_LOCK before this function returns.
      */
      rc = pagerLock(pPager, EXCLUSIVE_LOCK);
      if( rc!=SQLITE_OK ){
        rc = pager_error(pPager, rc);
        goto failed;
      }
 
      /* If it is not already open and the file exists on disk, open the 
      ** journal for read/write access. Write access is required because 
      ** in exclusive-access mode the file descriptor will be kept open 
      ** and possibly used for a transaction later on. Also, write-access 







<







4410
4411
4412
4413
4414
4415
4416

4417
4418
4419
4420
4421
4422
4423
      ** on the database file.
      **
      ** Unless the pager is in locking_mode=exclusive mode, the lock is
      ** downgraded to SHARED_LOCK before this function returns.
      */
      rc = pagerLock(pPager, EXCLUSIVE_LOCK);
      if( rc!=SQLITE_OK ){

        goto failed;
      }
 
      /* If it is not already open and the file exists on disk, open the 
      ** journal for read/write access. Write access is required because 
      ** in exclusive-access mode the file descriptor will be kept open 
      ** and possibly used for a transaction later on. Also, write-access 
4407
4408
4409
4410
4411
4412
4413




4414
4415
4416
4417
4418
4419
4420
4421
4422
4423
4424
4425
4426
4427
4428
4429
4430
      /* Playback and delete the journal.  Drop the database write
      ** lock and reacquire the read lock. Purge the cache before
      ** playing back the hot-journal so that we don't end up with
      ** an inconsistent cache.  Sync the hot journal before playing
      ** it back since the process that crashed and left the hot journal
      ** probably did not sync it and we are required to always sync
      ** the journal before playing it back.




      */
      if( isOpen(pPager->jfd) ){
        rc = pagerSyncHotJournal(pPager);
        if( rc==SQLITE_OK ){
          pPager->eState = PAGER_WRITER_FINISHED;
          rc = pager_playback(pPager, 1);
          pPager->eState = PAGER_NONE;
        }
        if( rc!=SQLITE_OK ){
          rc = pager_error(pPager, rc);
          goto failed;
        }
      }else{
        osUnlock(pPager, SHARED_LOCK);
      }

      assert( pPager->eState==PAGER_NONE );







>
>
>
>









<







4462
4463
4464
4465
4466
4467
4468
4469
4470
4471
4472
4473
4474
4475
4476
4477
4478
4479
4480
4481

4482
4483
4484
4485
4486
4487
4488
      /* Playback and delete the journal.  Drop the database write
      ** lock and reacquire the read lock. Purge the cache before
      ** playing back the hot-journal so that we don't end up with
      ** an inconsistent cache.  Sync the hot journal before playing
      ** it back since the process that crashed and left the hot journal
      ** probably did not sync it and we are required to always sync
      ** the journal before playing it back.
      **
      ** Even if an error occurs while syncing or rolling back the 
      ** hot-journal, there is no need to enter the ERROR state here, as
      ** the pager never left state NONE anyhow.
      */
      if( isOpen(pPager->jfd) ){
        rc = pagerSyncHotJournal(pPager);
        if( rc==SQLITE_OK ){
          pPager->eState = PAGER_WRITER_FINISHED;
          rc = pager_playback(pPager, 1);
          pPager->eState = PAGER_NONE;
        }
        if( rc!=SQLITE_OK ){

          goto failed;
        }
      }else{
        osUnlock(pPager, SHARED_LOCK);
      }

      assert( pPager->eState==PAGER_NONE );
4484
4485
4486
4487
4488
4489
4490
4491
4492



4493
4494
4495
4496
4497
4498
4499
  if( pPager->eState==PAGER_NONE && rc==SQLITE_OK ){
    pPager->eState = PAGER_NONE;
    rc = pagerPagecount(pPager, &pPager->dbSize);
  }

 failed:
  if( rc!=SQLITE_OK ){
    /* pager_unlock() is a no-op for exclusive mode and in-memory databases. */
    pager_unlock(pPager);



  }else{
    pPager->eState = PAGER_READER;
  }
  return rc;
}

/*







<

>
>
>







4542
4543
4544
4545
4546
4547
4548

4549
4550
4551
4552
4553
4554
4555
4556
4557
4558
4559
  if( pPager->eState==PAGER_NONE && rc==SQLITE_OK ){
    pPager->eState = PAGER_NONE;
    rc = pagerPagecount(pPager, &pPager->dbSize);
  }

 failed:
  if( rc!=SQLITE_OK ){

    pager_unlock(pPager);
    assert( (pPager->eState==PAGER_NONE)
        ||  (pPager->exclusiveMode && pagerUseWal(pPager))
    );
  }else{
    pPager->eState = PAGER_READER;
  }
  return rc;
}

/*
5540
5541
5542
5543
5544
5545
5546
5547
5548
5549
5550
5551
5552
5553
5554
5555
5556
5557
5558
5559
5560
5561
5562
5563
5564
5565
5566
5567
5568
  int rc = SQLITE_OK;                  /* Return code */

  /* This routine should not be called if a prior error has occurred.
  ** But if (due to a coding error elsewhere in the system) it does get
  ** called, just return the same error code without doing anything. */
  if( NEVER(pPager->errCode) ) return pPager->errCode;

  /* This function should not be called if the pager is not in at least
  ** PAGER_RESERVED state. **FIXME**: Make it so that this test always
  ** fails - make it so that we never reach this point if we do not hold
  ** all necessary locks.
  */
  assert( pPager->eState==PAGER_WRITER_INITIAL
       || pPager->eState==PAGER_WRITER_FINISHED
       || (pagerUseWal(pPager) && pPager->eState==PAGER_WRITER_CACHEMOD)
  );
  assert( assert_pager_state(pPager) );

#if 0
  if( NEVER(pPager->state<PAGER_RESERVED) ) return SQLITE_ERROR;
#endif

  /* An optimization. If the database was not actually modified during
  ** this transaction, the pager is running in exclusive-mode and is
  ** using persistent journals, then this function is a no-op.
  **
  ** The start of the journal file currently contains a single journal 
  ** header with the nRec field set to 0. If such a journal is used as
  ** a hot-journal during hot-journal rollback, 0 changes will be made







<
<
<
<
<






<
<
<
<







5600
5601
5602
5603
5604
5605
5606





5607
5608
5609
5610
5611
5612




5613
5614
5615
5616
5617
5618
5619
  int rc = SQLITE_OK;                  /* Return code */

  /* This routine should not be called if a prior error has occurred.
  ** But if (due to a coding error elsewhere in the system) it does get
  ** called, just return the same error code without doing anything. */
  if( NEVER(pPager->errCode) ) return pPager->errCode;






  assert( pPager->eState==PAGER_WRITER_INITIAL
       || pPager->eState==PAGER_WRITER_FINISHED
       || (pagerUseWal(pPager) && pPager->eState==PAGER_WRITER_CACHEMOD)
  );
  assert( assert_pager_state(pPager) );





  /* An optimization. If the database was not actually modified during
  ** this transaction, the pager is running in exclusive-mode and is
  ** using persistent journals, then this function is a no-op.
  **
  ** The start of the journal file currently contains a single journal 
  ** header with the nRec field set to 0. If such a journal is used as
  ** a hot-journal during hot-journal rollback, 0 changes will be made
5633
5634
5635
5636
5637
5638
5639
5640
5641
5642
5643
5644
5645
5646
5647
5648
5649
5650
5651
5652
5653
5654
5655
5656
5657
5658
5659
5660
5661
5662
5663

5664
5665
5666
5667
5668
5669
5670
5671
5672
5673
5674
5675
5676
5677

  /* PagerRollback() is a no-op if called in READER or NONE state. */
  assert( assert_pager_state(pPager) );
  if( pPager->eState<=PAGER_READER ) return SQLITE_OK;

  if( pagerUseWal(pPager) ){
    int rc2;

    rc = sqlite3PagerSavepoint(pPager, SAVEPOINT_ROLLBACK, -1);
    rc2 = pager_end_transaction(pPager, pPager->setMaster);
    if( rc==SQLITE_OK ) rc = rc2;
    rc = pager_error(pPager, rc);
  }else if( pPager->eState==PAGER_WRITER_INITIAL ){
    rc = pager_end_transaction(pPager, pPager->setMaster);
  }else if( pPager->errCode && pPager->errCode!=SQLITE_FULL ){
    if( pPager->eState>=PAGER_WRITER_DBMOD ){
      pager_playback(pPager, 0);
    }
    rc = pPager->errCode;
  }else{
    if( pPager->eState==PAGER_WRITER_CACHEMOD ){
      int rc2;
      rc = pager_playback(pPager, 0);
      rc2 = pager_end_transaction(pPager, pPager->setMaster);
      if( rc==SQLITE_OK ){
        rc = rc2;
      }
    }else{
      rc = pager_playback(pPager, 0);
    }


    /* If an error occurs during a ROLLBACK, we can no longer trust the pager
    ** cache. So call pager_error() on the way out to make any error 
    ** persistent.
    */
    rc = pager_error(pPager, rc);
  }
  return rc;
}

/*
** Return TRUE if the database file is opened read-only.  Return FALSE
** if the database is (in theory) writable.
*/
u8 sqlite3PagerIsreadonly(Pager *pPager){







<



<

|
















|
>
|
|
<
|
|
<
<







5684
5685
5686
5687
5688
5689
5690

5691
5692
5693

5694
5695
5696
5697
5698
5699
5700
5701
5702
5703
5704
5705
5706
5707
5708
5709
5710
5711
5712
5713
5714
5715

5716
5717


5718
5719
5720
5721
5722
5723
5724

  /* PagerRollback() is a no-op if called in READER or NONE state. */
  assert( assert_pager_state(pPager) );
  if( pPager->eState<=PAGER_READER ) return SQLITE_OK;

  if( pagerUseWal(pPager) ){
    int rc2;

    rc = sqlite3PagerSavepoint(pPager, SAVEPOINT_ROLLBACK, -1);
    rc2 = pager_end_transaction(pPager, pPager->setMaster);
    if( rc==SQLITE_OK ) rc = rc2;

  }else if( pPager->eState==PAGER_WRITER_INITIAL ){
    rc = pager_end_transaction(pPager, 0);
  }else if( pPager->errCode && pPager->errCode!=SQLITE_FULL ){
    if( pPager->eState>=PAGER_WRITER_DBMOD ){
      pager_playback(pPager, 0);
    }
    rc = pPager->errCode;
  }else{
    if( pPager->eState==PAGER_WRITER_CACHEMOD ){
      int rc2;
      rc = pager_playback(pPager, 0);
      rc2 = pager_end_transaction(pPager, pPager->setMaster);
      if( rc==SQLITE_OK ){
        rc = rc2;
      }
    }else{
      rc = pager_playback(pPager, 0);
    }
  }

  /* If an error occurs during a ROLLBACK, we can no longer trust the pager
  ** cache. So call pager_error() on the way out to make any error persistent.

  */
  return pager_error(pPager, rc);


}

/*
** Return TRUE if the database file is opened read-only.  Return FALSE
** if the database is (in theory) writable.
*/
u8 sqlite3PagerIsreadonly(Pager *pPager){