Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Fix more bugs. The new rollback journal implementation now passes all regression tests under Linux. (CVS 867) |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
d10adc1c5727d76320d5919be55e86d0 |
User & Date: | drh 2003-02-16 19:13:37.000 |
Context
2003-02-16
| ||
22:21 | Added test code to check for file descriptor leaks. All regression tests pass now on both win2k and linux. (CVS 868) (check-in: 75ba78280f user: drh tags: trunk) | |
19:13 | Fix more bugs. The new rollback journal implementation now passes all regression tests under Linux. (CVS 867) (check-in: d10adc1c57 user: drh tags: trunk) | |
2003-02-15
| ||
23:09 | Fix the pager so that correct rollbacks occur when synchronous is turned off. This check-in also included unrelated documentation updates. (CVS 866) (check-in: 3ef0ad8a4f user: drh tags: trunk) | |
Changes
Changes to src/pager.c.
︙ | ︙ | |||
14 15 16 17 18 19 20 | ** The pager is used to access a database disk file. It implements ** atomic commit and rollback through the use of a journal file that ** is separate from the database file. The pager also implements file ** locking to prevent two processes from writing the same database ** file simultaneously, or one process from reading the database while ** another is writing. ** | | | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | ** The pager is used to access a database disk file. It implements ** atomic commit and rollback through the use of a journal file that ** is separate from the database file. The pager also implements file ** locking to prevent two processes from writing the same database ** file simultaneously, or one process from reading the database while ** another is writing. ** ** @(#) $Id: pager.c,v 1.78 2003/02/16 19:13:37 drh Exp $ */ #include "os.h" /* Must be first to enable large file support */ #include "sqliteInt.h" #include "pager.h" #include <assert.h> #include <string.h> |
︙ | ︙ | |||
480 481 482 483 484 485 486 | static int pager_playback_one_page(Pager *pPager, OsFile *jfd, int format){ int rc; PgHdr *pPg; /* An existing page in the cache */ PageRecord pgRec; u32 cksum; rc = read32bits(format, jfd, &pgRec.pgno); | | | | < | | | < | 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 | static int pager_playback_one_page(Pager *pPager, OsFile *jfd, int format){ int rc; PgHdr *pPg; /* An existing page in the cache */ PageRecord pgRec; u32 cksum; rc = read32bits(format, jfd, &pgRec.pgno); if( rc!=SQLITE_OK ) return rc; rc = sqliteOsRead(jfd, &pgRec.aData, sizeof(pgRec.aData)); if( rc!=SQLITE_OK ) return rc; /* Sanity checking on the page. This is more important that I originally ** thought. If a power failure occurs while the journal is being written, ** it could cause invalid data to be written into the journal. We need to ** detect this invalid data (with high probability) and ignore it. */ if( pgRec.pgno==0 ){ return SQLITE_DONE; } if( pgRec.pgno>pPager->dbSize ){ return SQLITE_OK; } if( format>=JOURNAL_FORMAT_3 ){ rc = read32bits(format, jfd, &cksum); if( rc ) return rc; if( pager_cksum(pPager, pgRec.pgno, pgRec.aData)!=cksum ){ return SQLITE_DONE; } } /* Playback the page. Update the in-memory copy of the page ** at the same time, if there is one. */ pPg = pager_lookup(pPager, pgRec.pgno); TRACE2("PLAYBACK %d\n", pgRec.pgno); sqliteOsSeek(&pPager->fd, (pgRec.pgno-1)*(off_t)SQLITE_PAGE_SIZE); rc = sqliteOsWrite(&pPager->fd, pgRec.aData, SQLITE_PAGE_SIZE); if( pPg ){ if( pPg->nRef==0 || memcmp(PGHDR_TO_DATA(pPg), pgRec.aData, SQLITE_PAGE_SIZE)==0 ){ /* Do not update the data on this page if the page is in use ** and the page has never been modified. This avoids resetting ** the "extra" data. That in turn avoids invalidating BTree cursors |
︙ | ︙ | |||
552 553 554 555 556 557 558 | ** journal file (as determined by looking at the magic number ** at the beginning) then this routine returns SQLITE_PROTOCOL. ** If any other errors occur during playback, the database will ** likely be corrupted, so the PAGER_ERR_CORRUPT bit is set in ** pPager->errMask and SQLITE_CORRUPT is returned. If it all ** works, then this routine returns SQLITE_OK. */ | | | 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 | ** journal file (as determined by looking at the magic number ** at the beginning) then this routine returns SQLITE_PROTOCOL. ** If any other errors occur during playback, the database will ** likely be corrupted, so the PAGER_ERR_CORRUPT bit is set in ** pPager->errMask and SQLITE_CORRUPT is returned. If it all ** works, then this routine returns SQLITE_OK. */ static int pager_playback(Pager *pPager, int useJournalSize){ off_t szJ; /* Size of the journal file in bytes */ int nRec; /* Number of Records in the journal */ int i; /* Loop counter */ Pgno mxPg = 0; /* Size of the original file in pages */ int format; /* Format of the journal file. */ unsigned char aMagic[sizeof(aJournalMagic1)]; int rc; |
︙ | ︙ | |||
597 598 599 600 601 602 603 | goto end_playback; } if( format>=JOURNAL_FORMAT_3 ){ rc = read32bits(format, &pPager->jfd, &nRec); if( rc ) goto end_playback; rc = read32bits(format, &pPager->jfd, &pPager->cksumInit); if( rc ) goto end_playback; | | | 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 | goto end_playback; } if( format>=JOURNAL_FORMAT_3 ){ rc = read32bits(format, &pPager->jfd, &nRec); if( rc ) goto end_playback; rc = read32bits(format, &pPager->jfd, &pPager->cksumInit); if( rc ) goto end_playback; if( nRec==0xffffffff || useJournalSize ){ nRec = (szJ - JOURNAL_HDR_SZ(3))/JOURNAL_PG_SZ(3); } }else{ nRec = (szJ - JOURNAL_HDR_SZ(2))/JOURNAL_PG_SZ(2); assert( nRec*JOURNAL_PG_SZ(2)+JOURNAL_HDR_SZ(2)==szJ ); } rc = read32bits(format, &pPager->jfd, &mxPg); |
︙ | ︙ | |||
1062 1063 1064 1065 1066 1067 1068 | off_t szJ; if( pPager->fullSync ){ TRACE1("SYNC\n"); rc = sqliteOsSync(&pPager->jfd); if( rc!=0 ) return rc; } sqliteOsSeek(&pPager->jfd, sizeof(aJournalMagic1)); | | > | 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 | off_t szJ; if( pPager->fullSync ){ TRACE1("SYNC\n"); rc = sqliteOsSync(&pPager->jfd); if( rc!=0 ) return rc; } sqliteOsSeek(&pPager->jfd, sizeof(aJournalMagic1)); rc = write32bits(&pPager->jfd, pPager->nRec); if( rc ) return rc; szJ = JOURNAL_HDR_SZ(journal_format) + pPager->nRec*JOURNAL_PG_SZ(journal_format); sqliteOsSeek(&pPager->jfd, szJ); } TRACE1("SYNC\n"); rc = sqliteOsSync(&pPager->jfd); if( rc!=0 ) return rc; |
︙ | ︙ | |||
1221 1222 1223 1224 1225 1226 1227 | } pPager->journalOpen = 1; pPager->journalStarted = 0; /* Playback and delete the journal. Drop the database write ** lock and reacquire the read lock. */ | | | 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 | } pPager->journalOpen = 1; pPager->journalStarted = 0; /* Playback and delete the journal. Drop the database write ** lock and reacquire the read lock. */ rc = pager_playback(pPager, 0); if( rc!=SQLITE_OK ){ return rc; } } pPg = 0; }else{ /* Search for page in cache */ |
︙ | ︙ | |||
1657 1658 1659 1660 1661 1662 1663 | szPg = SQLITE_PAGE_SIZE+4; } store32bits(pPg->pgno, pPg, -4); rc = sqliteOsWrite(&pPager->jfd, &((char*)pData)[-4], szPg); if( journal_format>=JOURNAL_FORMAT_3 ){ *(u32*)PGHDR_TO_EXTRA(pPg) = saved; } | < > | 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 | szPg = SQLITE_PAGE_SIZE+4; } store32bits(pPg->pgno, pPg, -4); rc = sqliteOsWrite(&pPager->jfd, &((char*)pData)[-4], szPg); if( journal_format>=JOURNAL_FORMAT_3 ){ *(u32*)PGHDR_TO_EXTRA(pPg) = saved; } if( rc!=SQLITE_OK ){ sqlitepager_rollback(pPager); pPager->errMask |= PAGER_ERR_FULL; return rc; } pPager->nRec++; assert( pPager->aInJournal!=0 ); pPager->aInJournal[pPg->pgno/8] |= 1<<(pPg->pgno&7); pPg->needSync = !pPager->noSync; pPg->inJournal = 1; if( pPager->ckptInUse ){ pPager->aInCkpt[pPg->pgno/8] |= 1<<(pPg->pgno&7); page_add_to_ckpt_list(pPg); |
︙ | ︙ | |||
1877 1878 1879 1880 1881 1882 1883 | rc = pager_unwritelock(pPager); pPager->dbSize = -1; return rc; } if( pPager->errMask!=0 && pPager->errMask!=PAGER_ERR_FULL ){ if( pPager->state>=SQLITE_WRITELOCK ){ | | | | 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 | rc = pager_unwritelock(pPager); pPager->dbSize = -1; return rc; } if( pPager->errMask!=0 && pPager->errMask!=PAGER_ERR_FULL ){ if( pPager->state>=SQLITE_WRITELOCK ){ pager_playback(pPager, 1); } return pager_errcode(pPager); } if( pPager->state!=SQLITE_WRITELOCK ){ return SQLITE_OK; } rc = pager_playback(pPager, 1); if( rc!=SQLITE_OK ){ rc = SQLITE_CORRUPT; pPager->errMask |= PAGER_ERR_CORRUPT; } pPager->dbSize = -1; return rc; } |
︙ | ︙ |
Changes to src/test1.c.
︙ | ︙ | |||
9 10 11 12 13 14 15 | ** May you share freely, never taking more than you give. ** ************************************************************************* ** Code for testing the printf() interface to SQLite. This code ** is not included in the SQLite library. It is used for automated ** testing of the SQLite library. ** | | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | ** May you share freely, never taking more than you give. ** ************************************************************************* ** Code for testing the printf() interface to SQLite. This code ** is not included in the SQLite library. It is used for automated ** testing of the SQLite library. ** ** $Id: test1.c,v 1.21 2003/02/16 19:13:37 drh Exp $ */ #include "sqliteInt.h" #include "tcl.h" #include <stdlib.h> #include <string.h> /* |
︙ | ︙ | |||
821 822 823 824 825 826 827 828 829 830 831 832 833 834 | sprintf(zBuf, "(%d) ", rc); Tcl_AppendResult(interp, zBuf, zErrMsg, 0); sqlite_freemem(zErrMsg); return TCL_ERROR; } return TCL_OK; } /* ** Register commands with the TCL interpreter. */ int Sqlitetest1_Init(Tcl_Interp *interp){ extern int sqlite_search_count; static struct { | > > > > > > > > > > > > > > > > > > > > > > | 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 | sprintf(zBuf, "(%d) ", rc); Tcl_AppendResult(interp, zBuf, zErrMsg, 0); sqlite_freemem(zErrMsg); return TCL_ERROR; } return TCL_OK; } /* ** Usage: breakpoint ** ** This routine exists for one purpose - to provide a place to put a ** breakpoint with GDB that can be triggered using TCL code. The use ** for this is when a particular test fails on (say) the 1485th iteration. ** In the TCL test script, we can add code like this: ** ** if {$i==1485} breakpoint ** ** Then run testfixture in the debugger and wait for the breakpoint to ** fire. Then additional breakpoints can be set to trace down the bug. */ static int test_breakpoint( void *NotUsed, Tcl_Interp *interp, /* The TCL interpreter that invoked this command */ int argc, /* Number of arguments */ char **argv /* Text of each argument */ ){ return TCL_OK; /* Do nothing */ } /* ** Register commands with the TCL interpreter. */ int Sqlitetest1_Init(Tcl_Interp *interp){ extern int sqlite_search_count; static struct { |
︙ | ︙ | |||
854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 | #ifdef MEMORY_DEBUG { "sqlite_malloc_fail", (Tcl_CmdProc*)sqlite_malloc_fail }, { "sqlite_malloc_stat", (Tcl_CmdProc*)sqlite_malloc_stat }, #endif { "sqlite_compile", (Tcl_CmdProc*)test_compile }, { "sqlite_step", (Tcl_CmdProc*)test_step }, { "sqlite_finalize", (Tcl_CmdProc*)test_finalize }, }; int i; for(i=0; i<sizeof(aCmd)/sizeof(aCmd[0]); i++){ Tcl_CreateCommand(interp, aCmd[i].zName, aCmd[i].xProc, 0, 0); } Tcl_LinkVar(interp, "sqlite_search_count", (char*)&sqlite_search_count, TCL_LINK_INT); return TCL_OK; } | > | 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 | #ifdef MEMORY_DEBUG { "sqlite_malloc_fail", (Tcl_CmdProc*)sqlite_malloc_fail }, { "sqlite_malloc_stat", (Tcl_CmdProc*)sqlite_malloc_stat }, #endif { "sqlite_compile", (Tcl_CmdProc*)test_compile }, { "sqlite_step", (Tcl_CmdProc*)test_step }, { "sqlite_finalize", (Tcl_CmdProc*)test_finalize }, { "breakpoint", (Tcl_CmdProc*)test_breakpoint }, }; int i; for(i=0; i<sizeof(aCmd)/sizeof(aCmd[0]); i++){ Tcl_CreateCommand(interp, aCmd[i].zName, aCmd[i].xProc, 0, 0); } Tcl_LinkVar(interp, "sqlite_search_count", (char*)&sqlite_search_count, TCL_LINK_INT); return TCL_OK; } |
Changes to test/pager.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 | # 2001 September 15 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # This file implements regression tests for SQLite library. The # focus of this script is page cache subsystem. # | | > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | # 2001 September 15 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # This file implements regression tests for SQLite library. The # focus of this script is page cache subsystem. # # $Id: pager.test,v 1.13 2003/02/16 19:13:37 drh Exp $ set testdir [file dirname $argv0] source $testdir/tester.tcl if {[info commands pager_open]!=""} { db close # Basic sanity check. Open and close a pager. # do_test pager-1.0 { catch {file delete -force ptf1.db} catch {file delete -force ptf1.db-journal} set v [catch { |
︙ | ︙ | |||
170 171 172 173 174 175 176 177 178 179 180 181 182 183 | do_test pager-2.28 { set v [catch { pager_rollback $::p1 } msg] lappend v $msg } {0 {}} do_test pager-2.29 { page_read $::g1 } {Page-One} do_test pager-2.99 { pager_close $::p1 } {} do_test pager-3.1 { | > > | 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | do_test pager-2.28 { set v [catch { pager_rollback $::p1 } msg] lappend v $msg } {0 {}} do_test pager-2.29 { page_unref $::g1 set ::g1 [page_get $::p1 1] page_read $::g1 } {Page-One} do_test pager-2.99 { pager_close $::p1 } {} do_test pager-3.1 { |
︙ | ︙ |