Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Pull over all the latest trunk changes. |
|---|---|
| Downloads: | Tarball | ZIP archive | SQL archive |
| Timelines: | family | ancestors | descendants | both | wal-trace-375 |
| Files: | files | file ages | folders |
| SHA1: |
ca86d04be158df89f474e5b82ce3418d |
| User & Date: | drh 2011-01-18 17:34:39 |
Context
|
2011-02-19
| ||
| 14:19 | Changes which attempt to address an obscure SQLITE_PROTOCOL error. (Leaf check-in: 1725aa75 user: drh tags: wal-trace-375) | |
|
2011-01-18
| ||
| 17:34 | Pull over all the latest trunk changes. (check-in: ca86d04b user: drh tags: wal-trace-375) | |
| 17:03 | Do not use mutexes in the pcache implementation unless SQLITE_ENABLE_MEMORY_MANAGMENT is defined. This is a performance enhancement. A side effect is that pcaches will not steal pages from one another unless ENABLE_MEMORY_MANAGEMENT is set, or unless SQLITE_THREADSAFE=0. (check-in: e5ca59e6 user: drh tags: trunk) | |
|
2011-01-05
| ||
| 12:50 | Add detailed error logging to WAL in an effort to track down an obscure SQLITE_PROTOCOL problem. This code is intended for debugging and not for release. (check-in: 2c2afdd0 user: drh tags: wal-trace-375) | |
Changes
Changes to VERSION.
1 -3.7.4 1 +3.7.5
Changes to configure.
1 1 #! /bin/sh 2 2 # Guess values for system-dependent variables and create Makefiles. 3 -# Generated by GNU Autoconf 2.62 for sqlite 3.7.4. 3 +# Generated by GNU Autoconf 2.62 for sqlite 3.7.5. 4 4 # 5 5 # Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001, 6 6 # 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software Foundation, Inc. 7 7 # This configure script is free software; the Free Software Foundation 8 8 # gives unlimited permission to copy, distribute and modify it. 9 9 ## --------------------- ## 10 10 ## M4sh Initialization. ## ................................................................................ 739 739 MFLAGS= 740 740 MAKEFLAGS= 741 741 SHELL=${CONFIG_SHELL-/bin/sh} 742 742 743 743 # Identity of this package. 744 744 PACKAGE_NAME='sqlite' 745 745 PACKAGE_TARNAME='sqlite' 746 -PACKAGE_VERSION='3.7.4' 747 -PACKAGE_STRING='sqlite 3.7.4' 746 +PACKAGE_VERSION='3.7.5' 747 +PACKAGE_STRING='sqlite 3.7.5' 748 748 PACKAGE_BUGREPORT='' 749 749 750 750 # Factoring default headers for most tests. 751 751 ac_includes_default="\ 752 752 #include <stdio.h> 753 753 #ifdef HAVE_SYS_TYPES_H 754 754 # include <sys/types.h> ................................................................................ 1481 1481 # 1482 1482 # Report the --help message. 1483 1483 # 1484 1484 if test "$ac_init_help" = "long"; then 1485 1485 # Omit some internal or obsolete options to make the list less imposing. 1486 1486 # This message is too long to be a string in the A/UX 3.1 sh. 1487 1487 cat <<_ACEOF 1488 -\`configure' configures sqlite 3.7.4 to adapt to many kinds of systems. 1488 +\`configure' configures sqlite 3.7.5 to adapt to many kinds of systems. 1489 1489 1490 1490 Usage: $0 [OPTION]... [VAR=VALUE]... 1491 1491 1492 1492 To assign environment variables (e.g., CC, CFLAGS...), specify them as 1493 1493 VAR=VALUE. See below for descriptions of some of the useful variables. 1494 1494 1495 1495 Defaults for the options are specified in brackets. ................................................................................ 1546 1546 --build=BUILD configure for building on BUILD [guessed] 1547 1547 --host=HOST cross-compile to build programs to run on HOST [BUILD] 1548 1548 _ACEOF 1549 1549 fi 1550 1550 1551 1551 if test -n "$ac_init_help"; then 1552 1552 case $ac_init_help in 1553 - short | recursive ) echo "Configuration of sqlite 3.7.4:";; 1553 + short | recursive ) echo "Configuration of sqlite 3.7.5:";; 1554 1554 esac 1555 1555 cat <<\_ACEOF 1556 1556 1557 1557 Optional Features: 1558 1558 --disable-option-checking ignore unrecognized --enable/--with options 1559 1559 --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no) 1560 1560 --enable-FEATURE[=ARG] include FEATURE [ARG=yes] ................................................................................ 1662 1662 cd "$ac_pwd" || { ac_status=$?; break; } 1663 1663 done 1664 1664 fi 1665 1665 1666 1666 test -n "$ac_init_help" && exit $ac_status 1667 1667 if $ac_init_version; then 1668 1668 cat <<\_ACEOF 1669 -sqlite configure 3.7.4 1669 +sqlite configure 3.7.5 1670 1670 generated by GNU Autoconf 2.62 1671 1671 1672 1672 Copyright (C) 1992, 1993, 1994, 1995, 1996, 1998, 1999, 2000, 2001, 1673 1673 2002, 2003, 2004, 2005, 2006, 2007, 2008 Free Software Foundation, Inc. 1674 1674 This configure script is free software; the Free Software Foundation 1675 1675 gives unlimited permission to copy, distribute and modify it. 1676 1676 _ACEOF 1677 1677 exit 1678 1678 fi 1679 1679 cat >config.log <<_ACEOF 1680 1680 This file contains any messages produced by compilers while 1681 1681 running configure, to aid debugging if configure makes a mistake. 1682 1682 1683 -It was created by sqlite $as_me 3.7.4, which was 1683 +It was created by sqlite $as_me 3.7.5, which was 1684 1684 generated by GNU Autoconf 2.62. Invocation command line was 1685 1685 1686 1686 $ $0 $@ 1687 1687 1688 1688 _ACEOF 1689 1689 exec 5>>config.log 1690 1690 { ................................................................................ 13938 13938 13939 13939 exec 6>&1 13940 13940 13941 13941 # Save the log message, to keep $[0] and so on meaningful, and to 13942 13942 # report actual input values of CONFIG_FILES etc. instead of their 13943 13943 # values after options handling. 13944 13944 ac_log=" 13945 -This file was extended by sqlite $as_me 3.7.4, which was 13945 +This file was extended by sqlite $as_me 3.7.5, which was 13946 13946 generated by GNU Autoconf 2.62. Invocation command line was 13947 13947 13948 13948 CONFIG_FILES = $CONFIG_FILES 13949 13949 CONFIG_HEADERS = $CONFIG_HEADERS 13950 13950 CONFIG_LINKS = $CONFIG_LINKS 13951 13951 CONFIG_COMMANDS = $CONFIG_COMMANDS 13952 13952 $ $0 $@ ................................................................................ 13991 13991 $config_commands 13992 13992 13993 13993 Report bugs to <bug-autoconf@gnu.org>." 13994 13994 13995 13995 _ACEOF 13996 13996 cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 13997 13997 ac_cs_version="\\ 13998 -sqlite config.status 3.7.4 13998 +sqlite config.status 3.7.5 13999 13999 configured by $0, generated by GNU Autoconf 2.62, 14000 14000 with options \\"`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`\\" 14001 14001 14002 14002 Copyright (C) 2008 Free Software Foundation, Inc. 14003 14003 This config.status script is free software; the Free Software Foundation 14004 14004 gives unlimited permission to copy, distribute and modify it." 14005 14005
Changes to ext/fts3/fts3_snippet.c.
952 952 sqlite3_int64 nDoc; 953 953 954 954 if( !*ppStmt ){ 955 955 int rc = sqlite3Fts3SelectDoctotal(pTab, ppStmt); 956 956 if( rc!=SQLITE_OK ) return rc; 957 957 } 958 958 pStmt = *ppStmt; 959 + assert( sqlite3_data_count(pStmt)==1 ); 959 960 960 961 a = sqlite3_column_blob(pStmt, 0); 961 962 a += sqlite3Fts3GetVarint(a, &nDoc); 962 963 *pnDoc = (u32)nDoc; 963 964 964 965 if( paLen ) *paLen = a; 965 966 return SQLITE_OK;
Changes to ext/fts3/fts3_write.c.
1098 1098 ** varints, where nCol is the number of columns in the FTS3 table. 1099 1099 ** The first varint is the number of documents currently stored in 1100 1100 ** the table. The following nCol varints contain the total amount of 1101 1101 ** data stored in all rows of each column of the table, from left 1102 1102 ** to right. 1103 1103 */ 1104 1104 sqlite3_stmt *pStmt; 1105 - rc = fts3SqlStmt(p, SQL_SELECT_DOCTOTAL, &pStmt, 0); 1106 - if( rc ) return rc; 1107 - if( sqlite3_step(pStmt)==SQLITE_ROW ){ 1108 1105 sqlite3_int64 nDoc = 0; 1109 1106 sqlite3_int64 nByte = 0; 1110 - const char *a = sqlite3_column_blob(pStmt, 0); 1107 + const char *a; 1108 + rc = sqlite3Fts3SelectDoctotal(p, &pStmt); 1109 + if( rc ) return rc; 1110 + a = sqlite3_column_blob(pStmt, 0); 1111 1111 if( a ){ 1112 1112 const char *pEnd = &a[sqlite3_column_bytes(pStmt, 0)]; 1113 1113 a += sqlite3Fts3GetVarint(a, &nDoc); 1114 1114 while( a<pEnd ){ 1115 1115 a += sqlite3Fts3GetVarint(a, &nByte); 1116 1116 } 1117 1117 } 1118 + if( nDoc==0 || nByte==0 ){ 1119 + sqlite3_reset(pStmt); 1120 + return SQLITE_CORRUPT; 1121 + } 1118 1122 1119 - pCsr->nRowAvg = (int)(((nByte / nDoc) + pgsz - 1) / pgsz); 1120 - } 1123 + pCsr->nRowAvg = (int)(((nByte / nDoc) + pgsz) / pgsz); 1124 + assert( pCsr->nRowAvg>0 ); 1121 1125 rc = sqlite3_reset(pStmt); 1122 - if( rc!=SQLITE_OK || pCsr->nRowAvg==0 ) return rc; 1126 + if( rc!=SQLITE_OK ) return rc; 1123 1127 } 1124 1128 1125 1129 /* Assume that a blob flows over onto overflow pages if it is larger 1126 1130 ** than (pgsz-35) bytes in size (the file-format documentation 1127 1131 ** confirms this). 1128 1132 */ 1129 1133 for(iBlock=pReader->iStartBlock; iBlock<=pReader->iLeafEndBlock; iBlock++){
install-sh became executable.
whitespace changes only
Changes to src/backup.c.
112 112 if( i<0 ){ 113 113 sqlite3Error(pErrorDb, SQLITE_ERROR, "unknown database %s", zDb); 114 114 return 0; 115 115 } 116 116 117 117 return pDb->aDb[i].pBt; 118 118 } 119 + 120 +/* 121 +** Attempt to set the page size of the destination to match the page size 122 +** of the source. 123 +*/ 124 +static int setDestPgsz(sqlite3_backup *p){ 125 + int rc; 126 + rc = sqlite3BtreeSetPageSize(p->pDest,sqlite3BtreeGetPageSize(p->pSrc),-1,0); 127 + return rc; 128 +} 119 129 120 130 /* 121 131 ** Create an sqlite3_backup process to copy the contents of zSrcDb from 122 132 ** connection handle pSrcDb to zDestDb in pDestDb. If successful, return 123 133 ** a pointer to the new sqlite3_backup object. 124 134 ** 125 135 ** If an error occurs, NULL is returned and an error code and error message ................................................................................ 166 176 p->pSrc = findBtree(pDestDb, pSrcDb, zSrcDb); 167 177 p->pDest = findBtree(pDestDb, pDestDb, zDestDb); 168 178 p->pDestDb = pDestDb; 169 179 p->pSrcDb = pSrcDb; 170 180 p->iNext = 1; 171 181 p->isAttached = 0; 172 182 173 - if( 0==p->pSrc || 0==p->pDest ){ 174 - /* One (or both) of the named databases did not exist. An error has 175 - ** already been written into the pDestDb handle. All that is left 176 - ** to do here is free the sqlite3_backup structure. 183 + if( 0==p->pSrc || 0==p->pDest || setDestPgsz(p)==SQLITE_NOMEM ){ 184 + /* One (or both) of the named databases did not exist or an OOM 185 + ** error was hit. The error has already been written into the 186 + ** pDestDb handle. All that is left to do here is free the 187 + ** sqlite3_backup structure. 177 188 */ 178 189 sqlite3_free(p); 179 190 p = 0; 180 191 } 181 192 } 182 193 if( p ){ 183 194 p->pSrc->nBackup++; ................................................................................ 426 437 ** 427 438 ** * Data stored on the pages immediately following the 428 439 ** pending-byte page in the source database may need to be 429 440 ** copied into the destination database. 430 441 */ 431 442 const i64 iSize = (i64)pgszSrc * (i64)nSrcPage; 432 443 sqlite3_file * const pFile = sqlite3PagerFile(pDestPager); 444 + i64 iOff; 445 + i64 iEnd; 433 446 434 447 assert( pFile ); 435 448 assert( (i64)nDestTruncate*(i64)pgszDest >= iSize || ( 436 449 nDestTruncate==(int)(PENDING_BYTE_PAGE(p->pDest->pBt)-1) 437 450 && iSize>=PENDING_BYTE && iSize<=PENDING_BYTE+pgszDest 438 451 )); 439 - if( SQLITE_OK==(rc = sqlite3PagerCommitPhaseOne(pDestPager, 0, 1)) 440 - && SQLITE_OK==(rc = backupTruncateFile(pFile, iSize)) 441 - && SQLITE_OK==(rc = sqlite3PagerSync(pDestPager)) 442 - ){ 443 - i64 iOff; 444 - i64 iEnd = MIN(PENDING_BYTE + pgszDest, iSize); 452 + iEnd = MIN(PENDING_BYTE + pgszDest, iSize); 445 453 for( 446 454 iOff=PENDING_BYTE+pgszSrc; 447 455 rc==SQLITE_OK && iOff<iEnd; 448 456 iOff+=pgszSrc 449 457 ){ 450 458 PgHdr *pSrcPg = 0; 451 459 const Pgno iSrcPg = (Pgno)((iOff/pgszSrc)+1); ................................................................................ 452 460 rc = sqlite3PagerGet(pSrcPager, iSrcPg, &pSrcPg); 453 461 if( rc==SQLITE_OK ){ 454 462 u8 *zData = sqlite3PagerGetData(pSrcPg); 455 463 rc = sqlite3OsWrite(pFile, zData, pgszSrc, iOff); 456 464 } 457 465 sqlite3PagerUnref(pSrcPg); 458 466 } 467 + if( rc==SQLITE_OK ){ 468 + rc = sqlite3PagerCommitPhaseOne(pDestPager, 0, 1); 469 + if( rc==SQLITE_OK ){ 470 + rc = backupTruncateFile(pFile, iSize); 471 + } 459 472 } 460 473 }else{ 461 474 rc = sqlite3PagerCommitPhaseOne(pDestPager, 0, 0); 462 475 } 463 476 464 477 /* Finish committing the transaction to the destination database. */ 465 478 if( SQLITE_OK==rc
Changes to src/btree.c.
2382 2382 pBt->usableSize = usableSize; 2383 2383 pBt->pageSize = pageSize; 2384 2384 freeTempSpace(pBt); 2385 2385 rc = sqlite3PagerSetPagesize(pBt->pPager, &pBt->pageSize, 2386 2386 pageSize-usableSize); 2387 2387 return rc; 2388 2388 } 2389 - if( nPageHeader>nPageFile ){ 2389 + if( (pBt->db->flags & SQLITE_RecoveryMode)==0 && nPageHeader>nPageFile ){ 2390 2390 rc = SQLITE_CORRUPT_BKPT; 2391 2391 goto page1_init_failed; 2392 2392 } 2393 2393 if( usableSize<480 ){ 2394 2394 goto page1_init_failed; 2395 2395 } 2396 2396 pBt->pageSize = pageSize;
Changes to src/btreeInt.h.
410 410 u8 secureDelete; /* True if secure_delete is enabled */ 411 411 u8 initiallyEmpty; /* Database is empty at start of transaction */ 412 412 u8 openFlags; /* Flags to sqlite3BtreeOpen() */ 413 413 #ifndef SQLITE_OMIT_AUTOVACUUM 414 414 u8 autoVacuum; /* True if auto-vacuum is enabled */ 415 415 u8 incrVacuum; /* True if incr-vacuum is enabled */ 416 416 #endif 417 + u8 inTransaction; /* Transaction state */ 418 + u8 doNotUseWAL; /* If true, do not open write-ahead-log file */ 417 419 u16 maxLocal; /* Maximum local payload in non-LEAFDATA tables */ 418 420 u16 minLocal; /* Minimum local payload in non-LEAFDATA tables */ 419 421 u16 maxLeaf; /* Maximum local payload in a LEAFDATA table */ 420 422 u16 minLeaf; /* Minimum local payload in a LEAFDATA table */ 421 - u8 inTransaction; /* Transaction state */ 422 - u8 doNotUseWAL; /* If true, do not open write-ahead-log file */ 423 423 u32 pageSize; /* Total number of bytes on a page */ 424 424 u32 usableSize; /* Number of usable bytes on each page */ 425 425 int nTransaction; /* Number of open transactions (read + write) */ 426 426 u32 nPage; /* Number of pages in the database */ 427 427 void *pSchema; /* Pointer to space allocated by sqlite3BtreeSchema() */ 428 428 void (*xFreeSchema)(void*); /* Destructor for BtShared.pSchema */ 429 429 sqlite3_mutex *mutex; /* Non-recursive mutex required to access this struct */ ................................................................................ 442 442 /* 443 443 ** An instance of the following structure is used to hold information 444 444 ** about a cell. The parseCellPtr() function fills in this structure 445 445 ** based on information extract from the raw disk page. 446 446 */ 447 447 typedef struct CellInfo CellInfo; 448 448 struct CellInfo { 449 - u8 *pCell; /* Pointer to the start of cell content */ 450 449 i64 nKey; /* The key for INTKEY tables, or number of bytes in key */ 450 + u8 *pCell; /* Pointer to the start of cell content */ 451 451 u32 nData; /* Number of bytes of data */ 452 452 u32 nPayload; /* Total amount of payload */ 453 453 u16 nHeader; /* Size of the cell content header in bytes */ 454 454 u16 nLocal; /* Amount of payload held locally */ 455 455 u16 iOverflow; /* Offset to overflow page number. Zero if no overflow */ 456 456 u16 nSize; /* Size of the cell content on the main b-tree page */ 457 457 }; ................................................................................ 485 485 Btree *pBtree; /* The Btree to which this cursor belongs */ 486 486 BtShared *pBt; /* The BtShared this cursor points to */ 487 487 BtCursor *pNext, *pPrev; /* Forms a linked list of all cursors */ 488 488 struct KeyInfo *pKeyInfo; /* Argument passed to comparison function */ 489 489 Pgno pgnoRoot; /* The root page of this tree */ 490 490 sqlite3_int64 cachedRowid; /* Next rowid cache. 0 means not valid */ 491 491 CellInfo info; /* A parse of the cell we are pointing at */ 492 + i64 nKey; /* Size of pKey, or last integer key */ 493 + void *pKey; /* Saved key that was cursor's last known position */ 494 + int skipNext; /* Prev() is noop if negative. Next() is noop if positive */ 492 495 u8 wrFlag; /* True if writable */ 493 496 u8 atLast; /* Cursor pointing to the last entry */ 494 497 u8 validNKey; /* True if info.nKey is valid */ 495 498 u8 eState; /* One of the CURSOR_XXX constants (see below) */ 496 - void *pKey; /* Saved key that was cursor's last known position */ 497 - i64 nKey; /* Size of pKey, or last integer key */ 498 - int skipNext; /* Prev() is noop if negative. Next() is noop if positive */ 499 499 #ifndef SQLITE_OMIT_INCRBLOB 500 - u8 isIncrblobHandle; /* True if this cursor is an incr. io handle */ 501 500 Pgno *aOverflow; /* Cache of overflow page locations */ 501 + u8 isIncrblobHandle; /* True if this cursor is an incr. io handle */ 502 502 #endif 503 503 i16 iPage; /* Index of current page in apPage */ 504 - MemPage *apPage[BTCURSOR_MAX_DEPTH]; /* Pages from root to current page */ 505 504 u16 aiIdx[BTCURSOR_MAX_DEPTH]; /* Current index in apPage[i] */ 505 + MemPage *apPage[BTCURSOR_MAX_DEPTH]; /* Pages from root to current page */ 506 506 }; 507 507 508 508 /* 509 509 ** Potential values for BtCursor.eState. 510 510 ** 511 511 ** CURSOR_VALID: 512 512 ** Cursor points to a valid entry. getPayload() etc. may be called.
Changes to src/ctime.c.
169 169 "OMIT_AUTOINCREMENT", 170 170 #endif 171 171 #ifdef SQLITE_OMIT_AUTOINIT 172 172 "OMIT_AUTOINIT", 173 173 #endif 174 174 #ifdef SQLITE_OMIT_AUTOMATIC_INDEX 175 175 "OMIT_AUTOMATIC_INDEX", 176 +#endif 177 +#ifdef SQLITE_OMIT_AUTORESET 178 + "OMIT_AUTORESET", 176 179 #endif 177 180 #ifdef SQLITE_OMIT_AUTOVACUUM 178 181 "OMIT_AUTOVACUUM", 179 182 #endif 180 183 #ifdef SQLITE_OMIT_BETWEEN_OPTIMIZATION 181 184 "OMIT_BETWEEN_OPTIMIZATION", 182 185 #endif
Changes to src/pager.c.
2745 2745 pPager->changeCountDone = pPager->tempFile; 2746 2746 2747 2747 if( rc==SQLITE_OK ){ 2748 2748 zMaster = pPager->pTmpSpace; 2749 2749 rc = readMasterJournal(pPager->jfd, zMaster, pPager->pVfs->mxPathname+1); 2750 2750 testcase( rc!=SQLITE_OK ); 2751 2751 } 2752 - if( rc==SQLITE_OK && !pPager->noSync 2752 + if( rc==SQLITE_OK 2753 2753 && (pPager->eState>=PAGER_WRITER_DBMOD || pPager->eState==PAGER_OPEN) 2754 2754 ){ 2755 - rc = sqlite3OsSync(pPager->fd, pPager->syncFlags); 2755 + rc = sqlite3PagerSync(pPager); 2756 2756 } 2757 2757 if( rc==SQLITE_OK ){ 2758 2758 rc = pager_end_transaction(pPager, zMaster[0]!='\0'); 2759 2759 testcase( rc!=SQLITE_OK ); 2760 2760 } 2761 2761 if( rc==SQLITE_OK && zMaster[0] && res ){ 2762 2762 /* If there was a master journal and this routine will return success, ................................................................................ 2911 2911 rc = pagerUndoCallback((void *)pPager, pList->pgno); 2912 2912 pList = pNext; 2913 2913 } 2914 2914 2915 2915 return rc; 2916 2916 } 2917 2917 2918 + 2919 +/* 2920 +** Update the value of the change-counter at offsets 24 and 92 in 2921 +** the header and the sqlite version number at offset 96. 2922 +** 2923 +** This is an unconditional update. See also the pager_incr_changecounter() 2924 +** routine which only updates the change-counter if the update is actually 2925 +** needed, as determined by the pPager->changeCountDone state variable. 2926 +*/ 2927 +static void pager_write_changecounter(PgHdr *pPg){ 2928 + u32 change_counter; 2929 + 2930 + /* Increment the value just read and write it back to byte 24. */ 2931 + change_counter = sqlite3Get4byte((u8*)pPg->pPager->dbFileVers)+1; 2932 + put32bits(((char*)pPg->pData)+24, change_counter); 2933 + 2934 + /* Also store the SQLite version number in bytes 96..99 and in 2935 + ** bytes 92..95 store the change counter for which the version number 2936 + ** is valid. */ 2937 + put32bits(((char*)pPg->pData)+92, change_counter); 2938 + put32bits(((char*)pPg->pData)+96, SQLITE_VERSION_NUMBER); 2939 +} 2940 + 2918 2941 /* 2919 2942 ** This function is a wrapper around sqlite3WalFrames(). As well as logging 2920 2943 ** the contents of the list of pages headed by pList (connected by pDirty), 2921 2944 ** this function notifies any active backup processes that the pages have 2922 2945 ** changed. 2946 +** 2947 +** The list of pages passed into this routine is always sorted by page number. 2948 +** Hence, if page 1 appears anywhere on the list, it will be the first page. 2923 2949 */ 2924 2950 static int pagerWalFrames( 2925 2951 Pager *pPager, /* Pager object */ 2926 2952 PgHdr *pList, /* List of frames to log */ 2927 2953 Pgno nTruncate, /* Database size after this commit */ 2928 2954 int isCommit, /* True if this is a commit */ 2929 2955 int syncFlags /* Flags to pass to OsSync() (or 0) */ 2930 2956 ){ 2931 2957 int rc; /* Return code */ 2958 +#if defined(SQLITE_DEBUG) || defined(SQLITE_CHECK_PAGES) 2959 + PgHdr *p; /* For looping over pages */ 2960 +#endif 2932 2961 2933 2962 assert( pPager->pWal ); 2963 +#ifdef SQLITE_DEBUG 2964 + /* Verify that the page list is in accending order */ 2965 + for(p=pList; p && p->pDirty; p=p->pDirty){ 2966 + assert( p->pgno < p->pDirty->pgno ); 2967 + } 2968 +#endif 2969 + 2970 + if( pList->pgno==1 ) pager_write_changecounter(pList); 2934 2971 rc = sqlite3WalFrames(pPager->pWal, 2935 2972 pPager->pageSize, pList, nTruncate, isCommit, syncFlags 2936 2973 ); 2937 2974 if( rc==SQLITE_OK && pPager->pBackup ){ 2938 2975 PgHdr *p; 2939 2976 for(p=pList; p; p=p->pDirty){ 2940 2977 sqlite3BackupUpdate(pPager->pBackup, p->pgno, (u8 *)p->pData); 2941 2978 } 2942 2979 } 2943 2980 2944 2981 #ifdef SQLITE_CHECK_PAGES 2945 - { 2946 - PgHdr *p; 2947 - for(p=pList; p; p=p->pDirty) pager_set_pagehash(p); 2982 + for(p=pList; p; p=p->pDirty){ 2983 + pager_set_pagehash(p); 2948 2984 } 2949 2985 #endif 2950 2986 2951 2987 return rc; 2952 2988 } 2953 2989 2954 2990 /* ................................................................................ 3965 4001 ** set (set by sqlite3PagerDontWrite()). 3966 4002 */ 3967 4003 if( pgno<=pPager->dbSize && 0==(pList->flags&PGHDR_DONT_WRITE) ){ 3968 4004 i64 offset = (pgno-1)*(i64)pPager->pageSize; /* Offset to write */ 3969 4005 char *pData; /* Data to write */ 3970 4006 3971 4007 assert( (pList->flags&PGHDR_NEED_SYNC)==0 ); 4008 + if( pList->pgno==1 ) pager_write_changecounter(pList); 3972 4009 3973 4010 /* Encode the database */ 3974 4011 CODEC2(pPager, pList->pData, pgno, 6, return SQLITE_NOMEM, pData); 3975 4012 3976 4013 /* Write out the page data. */ 3977 4014 rc = sqlite3OsWrite(pPager->fd, pData, pPager->pageSize, offset); 3978 4015 ................................................................................ 5485 5522 pager_set_pagehash(pPg); 5486 5523 } 5487 5524 } 5488 5525 5489 5526 /* 5490 5527 ** This routine is called to increment the value of the database file 5491 5528 ** change-counter, stored as a 4-byte big-endian integer starting at 5492 -** byte offset 24 of the pager file. 5529 +** byte offset 24 of the pager file. The secondary change counter at 5530 +** 92 is also updated, as is the SQLite version number at offset 96. 5531 +** 5532 +** But this only happens if the pPager->changeCountDone flag is false. 5533 +** To avoid excess churning of page 1, the update only happens once. 5534 +** See also the pager_write_changecounter() routine that does an 5535 +** unconditional update of the change counters. 5493 5536 ** 5494 5537 ** If the isDirectMode flag is zero, then this is done by calling 5495 5538 ** sqlite3PagerWrite() on page 1, then modifying the contents of the 5496 5539 ** page data. In this case the file will be updated when the current 5497 5540 ** transaction is committed. 5498 5541 ** 5499 5542 ** The isDirectMode flag may only be non-zero if the library was compiled ................................................................................ 5526 5569 UNUSED_PARAMETER(isDirectMode); 5527 5570 #else 5528 5571 # define DIRECT_MODE isDirectMode 5529 5572 #endif 5530 5573 5531 5574 if( !pPager->changeCountDone && pPager->dbSize>0 ){ 5532 5575 PgHdr *pPgHdr; /* Reference to page 1 */ 5533 - u32 change_counter; /* Initial value of change-counter field */ 5534 5576 5535 5577 assert( !pPager->tempFile && isOpen(pPager->fd) ); 5536 5578 5537 5579 /* Open page 1 of the file for writing. */ 5538 5580 rc = sqlite3PagerGet(pPager, 1, &pPgHdr); 5539 5581 assert( pPgHdr==0 || rc==SQLITE_OK ); 5540 5582 ................................................................................ 5544 5586 ** above is always successful - hence the ALWAYS on rc==SQLITE_OK. 5545 5587 */ 5546 5588 if( !DIRECT_MODE && ALWAYS(rc==SQLITE_OK) ){ 5547 5589 rc = sqlite3PagerWrite(pPgHdr); 5548 5590 } 5549 5591 5550 5592 if( rc==SQLITE_OK ){ 5551 - /* Increment the value just read and write it back to byte 24. */ 5552 - change_counter = sqlite3Get4byte((u8*)pPager->dbFileVers); 5553 - change_counter++; 5554 - put32bits(((char*)pPgHdr->pData)+24, change_counter); 5555 - 5556 - /* Also store the SQLite version number in bytes 96..99 and in 5557 - ** bytes 92..95 store the change counter for which the version number 5558 - ** is valid. */ 5559 - put32bits(((char*)pPgHdr->pData)+92, change_counter); 5560 - put32bits(((char*)pPgHdr->pData)+96, SQLITE_VERSION_NUMBER); 5593 + /* Actually do the update of the change counter */ 5594 + pager_write_changecounter(pPgHdr); 5561 5595 5562 5596 /* If running in direct mode, write the contents of page 1 to the file. */ 5563 5597 if( DIRECT_MODE ){ 5564 5598 const void *zBuf; 5565 5599 assert( pPager->dbFileSize>0 ); 5566 5600 CODEC2(pPager, pPgHdr->pData, 1, 6, rc=SQLITE_NOMEM, zBuf); 5567 5601 if( rc==SQLITE_OK ){ ................................................................................ 5592 5626 int rc; /* Return code */ 5593 5627 assert( !MEMDB ); 5594 5628 if( pPager->noSync ){ 5595 5629 rc = SQLITE_OK; 5596 5630 }else{ 5597 5631 rc = sqlite3OsSync(pPager->fd, pPager->syncFlags); 5598 5632 } 5633 + if( isOpen(pPager->fd) ){ 5634 + sqlite3OsFileControl(pPager->fd, SQLITE_FCNTL_SYNC, (void *)&rc); 5635 + } 5599 5636 return rc; 5600 5637 } 5601 5638 5602 5639 /* 5603 5640 ** This function may only be called while a write-transaction is active in 5604 5641 ** rollback. If the connection is in WAL mode, this call is a no-op. 5605 5642 ** Otherwise, if the connection does not already have an EXCLUSIVE lock on ................................................................................ 5809 5846 Pgno nNew = pPager->dbSize - (pPager->dbSize==PAGER_MJ_PGNO(pPager)); 5810 5847 assert( pPager->eState==PAGER_WRITER_DBMOD ); 5811 5848 rc = pager_truncate(pPager, nNew); 5812 5849 if( rc!=SQLITE_OK ) goto commit_phase_one_exit; 5813 5850 } 5814 5851 5815 5852 /* Finally, sync the database file. */ 5816 - if( !pPager->noSync && !noSync ){ 5817 - rc = sqlite3OsSync(pPager->fd, pPager->syncFlags); 5853 + if( !noSync ){ 5854 + rc = sqlite3PagerSync(pPager); 5818 5855 } 5819 5856 IOTRACE(("DBSYNC %p\n", pPager)) 5820 5857 } 5821 5858 } 5822 5859 5823 5860 commit_phase_one_exit: 5824 5861 if( rc==SQLITE_OK && !pagerUseWal(pPager) ){ ................................................................................ 5922 5959 5923 5960 if( pagerUseWal(pPager) ){ 5924 5961 int rc2; 5925 5962 rc = sqlite3PagerSavepoint(pPager, SAVEPOINT_ROLLBACK, -1); 5926 5963 rc2 = pager_end_transaction(pPager, pPager->setMaster); 5927 5964 if( rc==SQLITE_OK ) rc = rc2; 5928 5965 }else if( !isOpen(pPager->jfd) || pPager->eState==PAGER_WRITER_LOCKED ){ 5966 + int eState = pPager->eState; 5929 5967 rc = pager_end_transaction(pPager, 0); 5968 + if( !MEMDB && eState>PAGER_WRITER_LOCKED ){ 5969 + /* This can happen using journal_mode=off. Move the pager to the error 5970 + ** state to indicate that the contents of the cache may not be trusted. 5971 + ** Any active readers will get SQLITE_ABORT. 5972 + */ 5973 + pPager->errCode = SQLITE_ABORT; 5974 + pPager->eState = PAGER_ERROR; 5975 + return rc; 5976 + } 5930 5977 }else{ 5931 5978 rc = pager_playback(pPager, 0); 5932 5979 } 5933 5980 5934 5981 assert( pPager->eState==PAGER_READER || rc!=SQLITE_OK ); 5935 5982 assert( rc==SQLITE_OK || rc==SQLITE_FULL || (rc&0xFF)==SQLITE_IOERR ); 5936 5983
Changes to src/pcache1.c.
18 18 */ 19 19 20 20 #include "sqliteInt.h" 21 21 22 22 typedef struct PCache1 PCache1; 23 23 typedef struct PgHdr1 PgHdr1; 24 24 typedef struct PgFreeslot PgFreeslot; 25 +typedef struct PGroup PGroup; 26 + 27 +/* Each page cache (or PCache) belongs to a PGroup. A PGroup is a set 28 +** of one or more PCaches that are able to recycle each others unpinned 29 +** pages when they are under memory pressure. A PGroup is an instance of 30 +** the following object. 31 +** 32 +** This page cache implementation works in one of two modes: 33 +** 34 +** (1) Every PCache is the sole member of its own PGroup. There is 35 +** one PGroup per PCache. 36 +** 37 +** (2) There is a single global PGroup that all PCaches are a member 38 +** of. 39 +** 40 +** Mode 1 uses more memory (since PCache instances are not able to rob 41 +** unused pages from other PCaches) but it also operates without a mutex, 42 +** and is therefore often faster. Mode 2 requires a mutex in order to be 43 +** threadsafe, but is able recycle pages more efficient. 44 +** 45 +** For mode (1), PGroup.mutex is NULL. For mode (2) there is only a single 46 +** PGroup which is the pcache1.grp global variable and its mutex is 47 +** SQLITE_MUTEX_STATIC_LRU. 48 +*/ 49 +struct PGroup { 50 + sqlite3_mutex *mutex; /* MUTEX_STATIC_LRU or NULL */ 51 + int nMaxPage; /* Sum of nMax for purgeable caches */ 52 + int nMinPage; /* Sum of nMin for purgeable caches */ 53 + int nCurrentPage; /* Number of purgeable pages allocated */ 54 + PgHdr1 *pLruHead, *pLruTail; /* LRU list of unpinned pages */ 55 +}; 25 56 26 57 /* Each page cache is an instance of the following object. Every 27 58 ** open database file (including each in-memory database and each 28 59 ** temporary or transient database) has a single page cache which 29 60 ** is an instance of this object. 30 61 ** 31 62 ** Pointers to structures of this type are cast and returned as 32 63 ** opaque sqlite3_pcache* handles. 33 64 */ 34 65 struct PCache1 { 35 66 /* Cache configuration parameters. Page size (szPage) and the purgeable 36 67 ** flag (bPurgeable) are set when the cache is created. nMax may be 37 68 ** modified at any time by a call to the pcache1CacheSize() method. 38 - ** The global mutex must be held when accessing nMax. 69 + ** The PGroup mutex must be held when accessing nMax. 39 70 */ 71 + PGroup *pGroup; /* PGroup this cache belongs to */ 40 72 int szPage; /* Size of allocated pages in bytes */ 41 73 int bPurgeable; /* True if cache is purgeable */ 42 74 unsigned int nMin; /* Minimum number of pages reserved */ 43 75 unsigned int nMax; /* Configured "cache_size" value */ 44 76 45 77 /* Hash table of all pages. The following variables may only be accessed 46 - ** when the accessor is holding the global mutex (see pcache1EnterMutex() 47 - ** and pcache1LeaveMutex()). 78 + ** when the accessor is holding the PGroup mutex. 48 79 */ 49 80 unsigned int nRecyclable; /* Number of pages in the LRU list */ 50 81 unsigned int nPage; /* Total number of pages in apHash */ 51 82 unsigned int nHash; /* Number of slots in apHash[] */ 52 83 PgHdr1 **apHash; /* Hash table for fast lookup by key */ 53 84 54 85 unsigned int iMaxKey; /* Largest key seen since xTruncate() */ ................................................................................ 76 107 PgFreeslot *pNext; /* Next free slot */ 77 108 }; 78 109 79 110 /* 80 111 ** Global data used by this cache. 81 112 */ 82 113 static SQLITE_WSD struct PCacheGlobal { 83 - sqlite3_mutex *mutex; /* static mutex MUTEX_STATIC_LRU */ 114 + PGroup grp; /* The global PGroup for mode (2) */ 84 115 85 - int nMaxPage; /* Sum of nMaxPage for purgeable caches */ 86 - int nMinPage; /* Sum of nMinPage for purgeable caches */ 87 - int nCurrentPage; /* Number of purgeable pages allocated */ 88 - PgHdr1 *pLruHead, *pLruTail; /* LRU list of unpinned pages */ 89 - 90 - /* Variables related to SQLITE_CONFIG_PAGECACHE settings. */ 116 + /* Variables related to SQLITE_CONFIG_PAGECACHE settings. The 117 + ** szSlot, nSlot, pStart, pEnd, nReserve, and isInit values are all 118 + ** fixed at sqlite3_initialize() time and do not require mutex protection. 119 + ** The nFreeSlot and pFree values do require mutex protection. 120 + */ 121 + int isInit; /* True if initialized */ 91 122 int szSlot; /* Size of each free slot */ 92 123 int nSlot; /* The number of pcache slots */ 93 - int nFreeSlot; /* Number of unused pcache slots */ 94 124 int nReserve; /* Try to keep nFreeSlot above this */ 95 125 void *pStart, *pEnd; /* Bounds of pagecache malloc range */ 126 + /* Above requires no mutex. Use mutex below for variable that follow. */ 127 + sqlite3_mutex *mutex; /* Mutex for accessing the following: */ 128 + int nFreeSlot; /* Number of unused pcache slots */ 96 129 PgFreeslot *pFree; /* Free page blocks */ 97 - int isInit; /* True if initialized */ 130 + /* The following value requires a mutex to change. We skip the mutex on 131 + ** reading because (1) most platforms read a 32-bit integer atomically and 132 + ** (2) even if an incorrect value is read, no great harm is done since this 133 + ** is really just an optimization. */ 134 + int bUnderPressure; /* True if low on PAGECACHE memory */ 98 135 } pcache1_g; 99 136 100 137 /* 101 138 ** All code in this file should access the global structure above via the 102 139 ** alias "pcache1". This ensures that the WSD emulation is used when 103 140 ** compiling for systems that do not support real WSD. 104 141 */ ................................................................................ 116 153 ** 117 154 ** assert( PGHDR1_TO_PAGE(PAGE_TO_PGHDR1(pCache, X))==X ); 118 155 */ 119 156 #define PGHDR1_TO_PAGE(p) (void*)(((char*)p) - p->pCache->szPage) 120 157 #define PAGE_TO_PGHDR1(c, p) (PgHdr1*)(((char*)p) + c->szPage) 121 158 122 159 /* 123 -** Macros to enter and leave the global LRU mutex. 160 +** Macros to enter and leave the PCache LRU mutex. 124 161 */ 125 -#define pcache1EnterMutex() sqlite3_mutex_enter(pcache1.mutex) 126 -#define pcache1LeaveMutex() sqlite3_mutex_leave(pcache1.mutex) 162 +#define pcache1EnterMutex(X) sqlite3_mutex_enter((X)->mutex) 163 +#define pcache1LeaveMutex(X) sqlite3_mutex_leave((X)->mutex) 127 164 128 165 /******************************************************************************/ 129 166 /******** Page Allocation/SQLITE_CONFIG_PCACHE Related Functions **************/ 130 167 131 168 /* 132 169 ** This function is called during initialization if a static buffer is 133 170 ** supplied to use for the page-cache by passing the SQLITE_CONFIG_PAGECACHE 134 171 ** verb to sqlite3_config(). Parameter pBuf points to an allocation large 135 172 ** enough to contain 'n' buffers of 'sz' bytes each. 173 +** 174 +** This routine is called from sqlite3_initialize() and so it is guaranteed 175 +** to be serialized already. There is no need for further mutexing. 136 176 */ 137 177 void sqlite3PCacheBufferSetup(void *pBuf, int sz, int n){ 138 178 if( pcache1.isInit ){ 139 179 PgFreeslot *p; 140 180 sz = ROUNDDOWN8(sz); 141 181 pcache1.szSlot = sz; 142 182 pcache1.nSlot = pcache1.nFreeSlot = n; 143 183 pcache1.nReserve = n>90 ? 10 : (n/10 + 1); 144 184 pcache1.pStart = pBuf; 145 185 pcache1.pFree = 0; 186 + pcache1.bUnderPressure = 0; 146 187 while( n-- ){ 147 188 p = (PgFreeslot*)pBuf; 148 189 p->pNext = pcache1.pFree; 149 190 pcache1.pFree = p; 150 191 pBuf = (void*)&((char*)pBuf)[sz]; 151 192 } 152 193 pcache1.pEnd = pBuf; ................................................................................ 154 195 } 155 196 156 197 /* 157 198 ** Malloc function used within this file to allocate space from the buffer 158 199 ** configured using sqlite3_config(SQLITE_CONFIG_PAGECACHE) option. If no 159 200 ** such buffer exists or there is no space left in it, this function falls 160 201 ** back to sqlite3Malloc(). 202 +** 203 +** Multiple threads can run this routine at the same time. Global variables 204 +** in pcache1 need to be protected via mutex. 161 205 */ 162 206 static void *pcache1Alloc(int nByte){ 163 - void *p; 164 - assert( sqlite3_mutex_held(pcache1.mutex) ); 207 + void *p = 0; 208 + assert( sqlite3_mutex_notheld(pcache1.grp.mutex) ); 165 209 sqlite3StatusSet(SQLITE_STATUS_PAGECACHE_SIZE, nByte); 166 - if( nByte<=pcache1.szSlot && pcache1.pFree ){ 167 - assert( pcache1.isInit ); 210 + if( nByte<=pcache1.szSlot ){ 211 + sqlite3_mutex_enter(pcache1.mutex); 168 212 p = (PgHdr1 *)pcache1.pFree; 213 + if( p ){ 169 214 pcache1.pFree = pcache1.pFree->pNext; 170 215 pcache1.nFreeSlot--; 216 + pcache1.bUnderPressure = pcache1.nFreeSlot<pcache1.nReserve; 171 217 assert( pcache1.nFreeSlot>=0 ); 172 218 sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_USED, 1); 173 - }else{ 174 - 175 - /* Allocate a new buffer using sqlite3Malloc. Before doing so, exit the 176 - ** global pcache mutex and unlock the pager-cache object pCache. This is 177 - ** so that if the attempt to allocate a new buffer causes the the 178 - ** configured soft-heap-limit to be breached, it will be possible to 179 - ** reclaim memory from this pager-cache. 219 + } 220 + sqlite3_mutex_leave(pcache1.mutex); 221 + } 222 + if( p==0 ){ 223 + /* Memory is not available in the SQLITE_CONFIG_PAGECACHE pool. Get 224 + ** it from sqlite3Malloc instead. 180 225 */ 181 - pcache1LeaveMutex(); 182 226 p = sqlite3Malloc(nByte); 183 - pcache1EnterMutex(); 184 227 if( p ){ 185 228 int sz = sqlite3MallocSize(p); 186 229 sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_OVERFLOW, sz); 187 230 } 188 231 sqlite3MemdebugSetType(p, MEMTYPE_PCACHE); 189 232 } 190 233 return p; 191 234 } 192 235 193 236 /* 194 237 ** Free an allocated buffer obtained from pcache1Alloc(). 195 238 */ 196 239 static void pcache1Free(void *p){ 197 - assert( sqlite3_mutex_held(pcache1.mutex) ); 198 240 if( p==0 ) return; 199 241 if( p>=pcache1.pStart && p<pcache1.pEnd ){ 200 242 PgFreeslot *pSlot; 243 + sqlite3_mutex_enter(pcache1.mutex); 201 244 sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_USED, -1); 202 245 pSlot = (PgFreeslot*)p; 203 246 pSlot->pNext = pcache1.pFree; 204 247 pcache1.pFree = pSlot; 205 248 pcache1.nFreeSlot++; 249 + pcache1.bUnderPressure = pcache1.nFreeSlot<pcache1.nReserve; 206 250 assert( pcache1.nFreeSlot<=pcache1.nSlot ); 251 + sqlite3_mutex_leave(pcache1.mutex); 207 252 }else{ 208 253 int iSize; 209 254 assert( sqlite3MemdebugHasType(p, MEMTYPE_PCACHE) ); 210 255 sqlite3MemdebugSetType(p, MEMTYPE_HEAP); 211 256 iSize = sqlite3MallocSize(p); 212 257 sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_OVERFLOW, -iSize); 213 258 sqlite3_free(p); ................................................................................ 215 260 } 216 261 217 262 #ifdef SQLITE_ENABLE_MEMORY_MANAGEMENT 218 263 /* 219 264 ** Return the size of a pcache allocation 220 265 */ 221 266 static int pcache1MemSize(void *p){ 222 - assert( sqlite3_mutex_held(pcache1.mutex) ); 223 267 if( p>=pcache1.pStart && p<pcache1.pEnd ){ 224 268 return pcache1.szSlot; 225 269 }else{ 226 270 int iSize; 227 271 assert( sqlite3MemdebugHasType(p, MEMTYPE_PCACHE) ); 228 272 sqlite3MemdebugSetType(p, MEMTYPE_HEAP); 229 273 iSize = sqlite3MallocSize(p); ................................................................................ 239 283 static PgHdr1 *pcache1AllocPage(PCache1 *pCache){ 240 284 int nByte = sizeof(PgHdr1) + pCache->szPage; 241 285 void *pPg = pcache1Alloc(nByte); 242 286 PgHdr1 *p; 243 287 if( pPg ){ 244 288 p = PAGE_TO_PGHDR1(pCache, pPg); 245 289 if( pCache->bPurgeable ){ 246 - pcache1.nCurrentPage++; 290 + pCache->pGroup->nCurrentPage++; 247 291 } 248 292 }else{ 249 293 p = 0; 250 294 } 251 295 return p; 252 296 } 253 297 ................................................................................ 256 300 ** 257 301 ** The pointer is allowed to be NULL, which is prudent. But it turns out 258 302 ** that the current implementation happens to never call this routine 259 303 ** with a NULL pointer, so we mark the NULL test with ALWAYS(). 260 304 */ 261 305 static void pcache1FreePage(PgHdr1 *p){ 262 306 if( ALWAYS(p) ){ 263 - if( p->pCache->bPurgeable ){ 264 - pcache1.nCurrentPage--; 307 + PCache1 *pCache = p->pCache; 308 + if( pCache->bPurgeable ){ 309 + pCache->pGroup->nCurrentPage--; 265 310 } 266 311 pcache1Free(PGHDR1_TO_PAGE(p)); 267 312 } 268 313 } 269 314 270 315 /* 271 316 ** Malloc function used by SQLite to obtain space from the buffer configured 272 317 ** using sqlite3_config(SQLITE_CONFIG_PAGECACHE) option. If no such buffer 273 318 ** exists, this function falls back to sqlite3Malloc(). 274 319 */ 275 320 void *sqlite3PageMalloc(int sz){ 276 - void *p; 277 - pcache1EnterMutex(); 278 - p = pcache1Alloc(sz); 279 - pcache1LeaveMutex(); 280 - return p; 321 + return pcache1Alloc(sz); 281 322 } 282 323 283 324 /* 284 325 ** Free an allocated buffer obtained from sqlite3PageMalloc(). 285 326 */ 286 327 void sqlite3PageFree(void *p){ 287 - pcache1EnterMutex(); 288 328 pcache1Free(p); 289 - pcache1LeaveMutex(); 290 329 } 291 330 292 331 293 332 /* 294 333 ** Return true if it desirable to avoid allocating a new page cache 295 334 ** entry. 296 335 ** ................................................................................ 303 342 ** 304 343 ** Or, the heap is used for all page cache memory put the heap is 305 344 ** under memory pressure, then again it is desirable to avoid 306 345 ** allocating a new page cache entry in order to avoid stressing 307 346 ** the heap even further. 308 347 */ 309 348 static int pcache1UnderMemoryPressure(PCache1 *pCache){ 310 - assert( sqlite3_mutex_held(pcache1.mutex) ); 311 349 if( pcache1.nSlot && pCache->szPage<=pcache1.szSlot ){ 312 - return pcache1.nFreeSlot<pcache1.nReserve; 350 + return pcache1.bUnderPressure; 313 351 }else{ 314 352 return sqlite3HeapNearlyFull(); 315 353 } 316 354 } 317 355 318 356 /******************************************************************************/ 319 357 /******** General Implementation Functions ************************************/ 320 358 321 359 /* 322 360 ** This function is used to resize the hash table used by the cache passed 323 361 ** as the first argument. 324 362 ** 325 -** The global mutex must be held when this function is called. 363 +** The PCache mutex must be held when this function is called. 326 364 */ 327 365 static int pcache1ResizeHash(PCache1 *p){ 328 366 PgHdr1 **apNew; 329 367 unsigned int nNew; 330 368 unsigned int i; 331 369 332 - assert( sqlite3_mutex_held(pcache1.mutex) ); 370 + assert( sqlite3_mutex_held(p->pGroup->mutex) ); 333 371 334 372 nNew = p->nHash*2; 335 373 if( nNew<256 ){ 336 374 nNew = 256; 337 375 } 338 376 339 - pcache1LeaveMutex(); 377 + pcache1LeaveMutex(p->pGroup); 340 378 if( p->nHash ){ sqlite3BeginBenignMalloc(); } 341 379 apNew = (PgHdr1 **)sqlite3_malloc(sizeof(PgHdr1 *)*nNew); 342 380 if( p->nHash ){ sqlite3EndBenignMalloc(); } 343 - pcache1EnterMutex(); 381 + pcache1EnterMutex(p->pGroup); 344 382 if( apNew ){ 345 383 memset(apNew, 0, sizeof(PgHdr1 *)*nNew); 346 384 for(i=0; i<p->nHash; i++){ 347 385 PgHdr1 *pPage; 348 386 PgHdr1 *pNext = p->apHash[i]; 349 387 while( (pPage = pNext)!=0 ){ 350 388 unsigned int h = pPage->iKey % nNew; ................................................................................ 359 397 } 360 398 361 399 return (p->apHash ? SQLITE_OK : SQLITE_NOMEM); 362 400 } 363 401 364 402 /* 365 403 ** This function is used internally to remove the page pPage from the 366 -** global LRU list, if is part of it. If pPage is not part of the global 404 +** PGroup LRU list, if is part of it. If pPage is not part of the PGroup 367 405 ** LRU list, then this function is a no-op. 368 406 ** 369 -** The global mutex must be held when this function is called. 407 +** The PGroup mutex must be held when this function is called. 408 +** 409 +** If pPage is NULL then this routine is a no-op. 370 410 */ 371 411 static void pcache1PinPage(PgHdr1 *pPage){ 372 - assert( sqlite3_mutex_held(pcache1.mutex) ); 373 - if( pPage && (pPage->pLruNext || pPage==pcache1.pLruTail) ){ 412 + PCache1 *pCache; 413 + PGroup *pGroup; 414 + 415 + if( pPage==0 ) return; 416 + pCache = pPage->pCache; 417 + pGroup = pCache->pGroup; 418 + assert( sqlite3_mutex_held(pGroup->mutex) ); 419 + if( pPage->pLruNext || pPage==pGroup->pLruTail ){ 374 420 if( pPage->pLruPrev ){ 375 421 pPage->pLruPrev->pLruNext = pPage->pLruNext; 376 422 } 377 423 if( pPage->pLruNext ){ 378 424 pPage->pLruNext->pLruPrev = pPage->pLruPrev; 379 425 } 380 - if( pcache1.pLruHead==pPage ){ 381 - pcache1.pLruHead = pPage->pLruNext; 426 + if( pGroup->pLruHead==pPage ){ 427 + pGroup->pLruHead = pPage->pLruNext; 382 428 } 383 - if( pcache1.pLruTail==pPage ){ 384 - pcache1.pLruTail = pPage->pLruPrev; 429 + if( pGroup->pLruTail==pPage ){ 430 + pGroup->pLruTail = pPage->pLruPrev; 385 431 } 386 432 pPage->pLruNext = 0; 387 433 pPage->pLruPrev = 0; 388 434 pPage->pCache->nRecyclable--; 389 435 } 390 436 } 391 437 392 438 393 439 /* 394 440 ** Remove the page supplied as an argument from the hash table 395 441 ** (PCache1.apHash structure) that it is currently stored in. 396 442 ** 397 -** The global mutex must be held when this function is called. 443 +** The PGroup mutex must be held when this function is called. 398 444 */ 399 445 static void pcache1RemoveFromHash(PgHdr1 *pPage){ 400 446 unsigned int h; 401 447 PCache1 *pCache = pPage->pCache; 402 448 PgHdr1 **pp; 403 449 450 + assert( sqlite3_mutex_held(pCache->pGroup->mutex) ); 404 451 h = pPage->iKey % pCache->nHash; 405 452 for(pp=&pCache->apHash[h]; (*pp)!=pPage; pp=&(*pp)->pNext); 406 453 *pp = (*pp)->pNext; 407 454 408 455 pCache->nPage--; 409 456 } 410 457 411 458 /* 412 -** If there are currently more than pcache.nMaxPage pages allocated, try 413 -** to recycle pages to reduce the number allocated to pcache.nMaxPage. 459 +** If there are currently more than nMaxPage pages allocated, try 460 +** to recycle pages to reduce the number allocated to nMaxPage. 414 461 */ 415 -static void pcache1EnforceMaxPage(void){ 416 - assert( sqlite3_mutex_held(pcache1.mutex) ); 417 - while( pcache1.nCurrentPage>pcache1.nMaxPage && pcache1.pLruTail ){ 418 - PgHdr1 *p = pcache1.pLruTail; 462 +static void pcache1EnforceMaxPage(PGroup *pGroup){ 463 + assert( sqlite3_mutex_held(pGroup->mutex) ); 464 + while( pGroup->nCurrentPage>pGroup->nMaxPage && pGroup->pLruTail ){ 465 + PgHdr1 *p = pGroup->pLruTail; 466 + assert( p->pCache->pGroup==pGroup ); 419 467 pcache1PinPage(p); 420 468 pcache1RemoveFromHash(p); 421 469 pcache1FreePage(p); 422 470 } 423 471 } 424 472 425 473 /* 426 474 ** Discard all pages from cache pCache with a page number (key value) 427 475 ** greater than or equal to iLimit. Any pinned pages that meet this 428 476 ** criteria are unpinned before they are discarded. 429 477 ** 430 -** The global mutex must be held when this function is called. 478 +** The PCache mutex must be held when this function is called. 431 479 */ 432 480 static void pcache1TruncateUnsafe( 433 - PCache1 *pCache, 434 - unsigned int iLimit 481 + PCache1 *pCache, /* The cache to truncate */ 482 + unsigned int iLimit /* Drop pages with this pgno or larger */ 435 483 ){ 436 - TESTONLY( unsigned int nPage = 0; ) /* Used to assert pCache->nPage is correct */ 484 + TESTONLY( unsigned int nPage = 0; ) /* To assert pCache->nPage is correct */ 437 485 unsigned int h; 438 - assert( sqlite3_mutex_held(pcache1.mutex) ); 486 + assert( sqlite3_mutex_held(pCache->pGroup->mutex) ); 439 487 for(h=0; h<pCache->nHash; h++){ 440 488 PgHdr1 **pp = &pCache->apHash[h]; 441 489 PgHdr1 *pPage; 442 490 while( (pPage = *pp)!=0 ){ 443 491 if( pPage->iKey>=iLimit ){ 444 492 pCache->nPage--; 445 493 *pp = pPage->pNext; ................................................................................ 461 509 ** Implementation of the sqlite3_pcache.xInit method. 462 510 */ 463 511 static int pcache1Init(void *NotUsed){ 464 512 UNUSED_PARAMETER(NotUsed); 465 513 assert( pcache1.isInit==0 ); 466 514 memset(&pcache1, 0, sizeof(pcache1)); 467 515 if( sqlite3GlobalConfig.bCoreMutex ){ 468 - pcache1.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_LRU); 516 + pcache1.grp.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_LRU); 517 + pcache1.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_PMEM); 469 518 } 470 519 pcache1.isInit = 1; 471 520 return SQLITE_OK; 472 521 } 473 522 474 523 /* 475 524 ** Implementation of the sqlite3_pcache.xShutdown method. ................................................................................ 484 533 485 534 /* 486 535 ** Implementation of the sqlite3_pcache.xCreate method. 487 536 ** 488 537 ** Allocate a new cache. 489 538 */ 490 539 static sqlite3_pcache *pcache1Create(int szPage, int bPurgeable){ 491 - PCache1 *pCache; 540 + PCache1 *pCache; /* The newly created page cache */ 541 + PGroup *pGroup; /* The group the new page cache will belong to */ 542 + int sz; /* Bytes of memory required to allocate the new cache */ 492 543 493 - pCache = (PCache1 *)sqlite3_malloc(sizeof(PCache1)); 544 + /* 545 + ** The seperateCache variable is true if each PCache has its own private 546 + ** PGroup. In other words, separateCache is true for mode (1) where no 547 + ** mutexing is required. 548 + ** 549 + ** * Always use a unified cache (mode-2) if ENABLE_MEMORY_MANAGEMENT 550 + ** 551 + ** * Always use a unified cache in single-threaded applications 552 + ** 553 + ** * Otherwise (if multi-threaded and ENABLE_MEMORY_MANAGEMENT is off) 554 + ** use separate caches (mode-1) 555 + */ 556 +#if defined(SQLITE_ENABLE_MEMORY_MANAGEMENT) || SQLITE_THREADSAFE==0 557 + const int separateCache = 0; 558 +#else 559 + int separateCache = sqlite3GlobalConfig.bCoreMutex>0; 560 +#endif 561 + 562 + sz = sizeof(PCache1) + sizeof(PGroup)*separateCache; 563 + pCache = (PCache1 *)sqlite3_malloc(sz); 494 564 if( pCache ){ 495 - memset(pCache, 0, sizeof(PCache1)); 565 + memset(pCache, 0, sz); 566 + if( separateCache ){ 567 + pGroup = (PGroup*)&pCache[1]; 568 + }else{ 569 + pGroup = &pcache1_g.grp; 570 + } 571 + pCache->pGroup = pGroup; 496 572 pCache->szPage = szPage; 497 573 pCache->bPurgeable = (bPurgeable ? 1 : 0); 498 574 if( bPurgeable ){ 499 575 pCache->nMin = 10; 500 - pcache1EnterMutex(); 501 - pcache1.nMinPage += pCache->nMin; 502 - pcache1LeaveMutex(); 576 + pcache1EnterMutex(pGroup); 577 + pGroup->nMinPage += pCache->nMin; 578 + pcache1LeaveMutex(pGroup); 503 579 } 504 580 } 505 581 return (sqlite3_pcache *)pCache; 506 582 } 507 583 508 584 /* 509 585 ** Implementation of the sqlite3_pcache.xCachesize method. 510 586 ** 511 587 ** Configure the cache_size limit for a cache. 512 588 */ 513 589 static void pcache1Cachesize(sqlite3_pcache *p, int nMax){ 514 590 PCache1 *pCache = (PCache1 *)p; 515 591 if( pCache->bPurgeable ){ 516 - pcache1EnterMutex(); 517 - pcache1.nMaxPage += (nMax - pCache->nMax); 592 + PGroup *pGroup = pCache->pGroup; 593 + pcache1EnterMutex(pGroup); 594 + pGroup->nMaxPage += (nMax - pCache->nMax); 518 595 pCache->nMax = nMax; 519 - pcache1EnforceMaxPage(); 520 - pcache1LeaveMutex(); 596 + pcache1EnforceMaxPage(pGroup); 597 + pcache1LeaveMutex(pGroup); 521 598 } 522 599 } 523 600 524 601 /* 525 602 ** Implementation of the sqlite3_pcache.xPagecount method. 526 603 */ 527 604 static int pcache1Pagecount(sqlite3_pcache *p){ 528 605 int n; 529 - pcache1EnterMutex(); 530 - n = ((PCache1 *)p)->nPage; 531 - pcache1LeaveMutex(); 606 + PCache1 *pCache = (PCache1*)p; 607 + pcache1EnterMutex(pCache->pGroup); 608 + n = pCache->nPage; 609 + pcache1LeaveMutex(pCache->pGroup); 532 610 return n; 533 611 } 534 612 535 613 /* 536 614 ** Implementation of the sqlite3_pcache.xFetch method. 537 615 ** 538 616 ** Fetch a page by key value. ................................................................................ 585 663 ** proceed to step 5. 586 664 ** 587 665 ** 5. Otherwise, allocate and return a new page buffer. 588 666 */ 589 667 static void *pcache1Fetch(sqlite3_pcache *p, unsigned int iKey, int createFlag){ 590 668 unsigned int nPinned; 591 669 PCache1 *pCache = (PCache1 *)p; 670 + PGroup *pGroup = pCache->pGroup; 592 671 PgHdr1 *pPage = 0; 593 672 594 673 assert( pCache->bPurgeable || createFlag!=1 ); 595 - pcache1EnterMutex(); 674 + pcache1EnterMutex(pGroup); 596 675 if( createFlag==1 ) sqlite3BeginBenignMalloc(); 597 676 598 677 /* Search the hash table for an existing entry. */ 599 678 if( pCache->nHash>0 ){ 600 679 unsigned int h = iKey % pCache->nHash; 601 680 for(pPage=pCache->apHash[h]; pPage&&pPage->iKey!=iKey; pPage=pPage->pNext); 602 681 } ................................................................................ 605 684 pcache1PinPage(pPage); 606 685 goto fetch_out; 607 686 } 608 687 609 688 /* Step 3 of header comment. */ 610 689 nPinned = pCache->nPage - pCache->nRecyclable; 611 690 if( createFlag==1 && ( 612 - nPinned>=(pcache1.nMaxPage+pCache->nMin-pcache1.nMinPage) 691 + nPinned>=(pGroup->nMaxPage+pCache->nMin-pGroup->nMinPage) 613 692 || nPinned>=(pCache->nMax * 9 / 10) 614 693 || pcache1UnderMemoryPressure(pCache) 615 694 )){ 616 695 goto fetch_out; 617 696 } 618 697 619 698 if( pCache->nPage>=pCache->nHash && pcache1ResizeHash(pCache) ){ 620 699 goto fetch_out; 621 700 } 622 701 623 702 /* Step 4. Try to recycle a page buffer if appropriate. */ 624 - if( pCache->bPurgeable && pcache1.pLruTail && ( 703 + if( pCache->bPurgeable && pGroup->pLruTail && ( 625 704 (pCache->nPage+1>=pCache->nMax) 626 - || pcache1.nCurrentPage>=pcache1.nMaxPage 705 + || pGroup->nCurrentPage>=pGroup->nMaxPage 627 706 || pcache1UnderMemoryPressure(pCache) 628 707 )){ 629 - pPage = pcache1.pLruTail; 708 + pPage = pGroup->pLruTail; 630 709 pcache1RemoveFromHash(pPage); 631 710 pcache1PinPage(pPage); 632 711 if( pPage->pCache->szPage!=pCache->szPage ){ 633 712 pcache1FreePage(pPage); 634 713 pPage = 0; 635 714 }else{ 636 - pcache1.nCurrentPage -= (pPage->pCache->bPurgeable - pCache->bPurgeable); 715 + pGroup->nCurrentPage -= 716 + (pPage->pCache->bPurgeable - pCache->bPurgeable); 637 717 } 638 718 } 639 719 640 720 /* Step 5. If a usable page buffer has still not been found, 641 721 ** attempt to allocate a new one. 642 722 */ 643 723 if( !pPage ){ 724 + pcache1LeaveMutex(pGroup); 644 725 pPage = pcache1AllocPage(pCache); 726 + pcache1EnterMutex(pGroup); 645 727 } 646 728 647 729 if( pPage ){ 648 730 unsigned int h = iKey % pCache->nHash; 649 731 pCache->nPage++; 650 732 pPage->iKey = iKey; 651 733 pPage->pNext = pCache->apHash[h]; ................................................................................ 657 739 } 658 740 659 741 fetch_out: 660 742 if( pPage && iKey>pCache->iMaxKey ){ 661 743 pCache->iMaxKey = iKey; 662 744 } 663 745 if( createFlag==1 ) sqlite3EndBenignMalloc(); 664 - pcache1LeaveMutex(); 746 + pcache1LeaveMutex(pGroup); 665 747 return (pPage ? PGHDR1_TO_PAGE(pPage) : 0); 666 748 } 667 749 668 750 669 751 /* 670 752 ** Implementation of the sqlite3_pcache.xUnpin method. 671 753 ** 672 754 ** Mark a page as unpinned (eligible for asynchronous recycling). 673 755 */ 674 756 static void pcache1Unpin(sqlite3_pcache *p, void *pPg, int reuseUnlikely){ 675 757 PCache1 *pCache = (PCache1 *)p; 676 758 PgHdr1 *pPage = PAGE_TO_PGHDR1(pCache, pPg); 759 + PGroup *pGroup = pCache->pGroup; 677 760 678 761 assert( pPage->pCache==pCache ); 679 - pcache1EnterMutex(); 762 + pcache1EnterMutex(pGroup); 680 763 681 764 /* It is an error to call this function if the page is already 682 - ** part of the global LRU list. 765 + ** part of the PGroup LRU list. 683 766 */ 684 767 assert( pPage->pLruPrev==0 && pPage->pLruNext==0 ); 685 - assert( pcache1.pLruHead!=pPage && pcache1.pLruTail!=pPage ); 768 + assert( pGroup->pLruHead!=pPage && pGroup->pLruTail!=pPage ); 686 769 687 - if( reuseUnlikely || pcache1.nCurrentPage>pcache1.nMaxPage ){ 770 + if( reuseUnlikely || pGroup->nCurrentPage>pGroup->nMaxPage ){ 688 771 pcache1RemoveFromHash(pPage); 689 772 pcache1FreePage(pPage); 690 773 }else{ 691 - /* Add the page to the global LRU list. Normally, the page is added to 692 - ** the head of the list (last page to be recycled). However, if the 693 - ** reuseUnlikely flag passed to this function is true, the page is added 694 - ** to the tail of the list (first page to be recycled). 695 - */ 696 - if( pcache1.pLruHead ){ 697 - pcache1.pLruHead->pLruPrev = pPage; 698 - pPage->pLruNext = pcache1.pLruHead; 699 - pcache1.pLruHead = pPage; 774 + /* Add the page to the PGroup LRU list. */ 775 + if( pGroup->pLruHead ){ 776 + pGroup->pLruHead->pLruPrev = pPage; 777 + pPage->pLruNext = pGroup->pLruHead; 778 + pGroup->pLruHead = pPage; 700 779 }else{ 701 - pcache1.pLruTail = pPage; 702 - pcache1.pLruHead = pPage; 780 + pGroup->pLruTail = pPage; 781 + pGroup->pLruHead = pPage; 703 782 } 704 783 pCache->nRecyclable++; 705 784 } 706 785 707 - pcache1LeaveMutex(); 786 + pcache1LeaveMutex(pCache->pGroup); 708 787 } 709 788 710 789 /* 711 790 ** Implementation of the sqlite3_pcache.xRekey method. 712 791 */ 713 792 static void pcache1Rekey( 714 793 sqlite3_pcache *p, ................................................................................ 719 798 PCache1 *pCache = (PCache1 *)p; 720 799 PgHdr1 *pPage = PAGE_TO_PGHDR1(pCache, pPg); 721 800 PgHdr1 **pp; 722 801 unsigned int h; 723 802 assert( pPage->iKey==iOld ); 724 803 assert( pPage->pCache==pCache ); 725 804 726 - pcache1EnterMutex(); 805 + pcache1EnterMutex(pCache->pGroup); 727 806 728 807 h = iOld%pCache->nHash; 729 808 pp = &pCache->apHash[h]; 730 809 while( (*pp)!=pPage ){ 731 810 pp = &(*pp)->pNext; 732 811 } 733 812 *pp = pPage->pNext; ................................................................................ 736 815 pPage->iKey = iNew; 737 816 pPage->pNext = pCache->apHash[h]; 738 817 pCache->apHash[h] = pPage; 739 818 if( iNew>pCache->iMaxKey ){ 740 819 pCache->iMaxKey = iNew; 741 820 } 742 821 743 - pcache1LeaveMutex(); 822 + pcache1LeaveMutex(pCache->pGroup); 744 823 } 745 824 746 825 /* 747 826 ** Implementation of the sqlite3_pcache.xTruncate method. 748 827 ** 749 828 ** Discard all unpinned pages in the cache with a page number equal to 750 829 ** or greater than parameter iLimit. Any pinned pages with a page number 751 830 ** equal to or greater than iLimit are implicitly unpinned. 752 831 */ 753 832 static void pcache1Truncate(sqlite3_pcache *p, unsigned int iLimit){ 754 833 PCache1 *pCache = (PCache1 *)p; 755 - pcache1EnterMutex(); 834 + pcache1EnterMutex(pCache->pGroup); 756 835 if( iLimit<=pCache->iMaxKey ){ 757 836 pcache1TruncateUnsafe(pCache, iLimit); 758 837 pCache->iMaxKey = iLimit-1; 759 838 } 760 - pcache1LeaveMutex(); 839 + pcache1LeaveMutex(pCache->pGroup); 761 840 } 762 841 763 842 /* 764 843 ** Implementation of the sqlite3_pcache.xDestroy method. 765 844 ** 766 845 ** Destroy a cache allocated using pcache1Create(). 767 846 */ 768 847 static void pcache1Destroy(sqlite3_pcache *p){ 769 848 PCache1 *pCache = (PCache1 *)p; 849 + PGroup *pGroup = pCache->pGroup; 770 850 assert( pCache->bPurgeable || (pCache->nMax==0 && pCache->nMin==0) ); 771 - pcache1EnterMutex(); 851 + pcache1EnterMutex(pGroup); 772 852 pcache1TruncateUnsafe(pCache, 0); 773 - pcache1.nMaxPage -= pCache->nMax; 774 - pcache1.nMinPage -= pCache->nMin; 775 - pcache1EnforceMaxPage(); 776 - pcache1LeaveMutex(); 853 + pGroup->nMaxPage -= pCache->nMax; 854 + pGroup->nMinPage -= pCache->nMin; 855 + pcache1EnforceMaxPage(pGroup); 856 + pcache1LeaveMutex(pGroup); 777 857 sqlite3_free(pCache->apHash); 778 858 sqlite3_free(pCache); 779 859 } 780 860 781 861 /* 782 862 ** This function is called during initialization (sqlite3_initialize()) to 783 863 ** install the default pluggable cache module, assuming the user has not ................................................................................ 808 888 ** 809 889 ** nReq is the number of bytes of memory required. Once this much has 810 890 ** been released, the function returns. The return value is the total number 811 891 ** of bytes of memory released. 812 892 */ 813 893 int sqlite3PcacheReleaseMemory(int nReq){ 814 894 int nFree = 0; 895 + assert( sqlite3_mutex_notheld(pcache1.grp.mutex) ); 896 + assert( sqlite3_mutex_notheld(pcache1.mutex) ); 815 897 if( pcache1.pStart==0 ){ 816 898 PgHdr1 *p; 817 - pcache1EnterMutex(); 818 - while( (nReq<0 || nFree<nReq) && ((p=pcache1.pLruTail)!=0) ){ 899 + pcache1EnterMutex(&pcache1.grp); 900 + while( (nReq<0 || nFree<nReq) && ((p=pcache1.grp.pLruTail)!=0) ){ 819 901 nFree += pcache1MemSize(PGHDR1_TO_PAGE(p)); 820 902 pcache1PinPage(p); 821 903 pcache1RemoveFromHash(p); 822 904 pcache1FreePage(p); 823 905 } 824 - pcache1LeaveMutex(); 906 + pcache1LeaveMutex(&pcache1.grp); 825 907 } 826 908 return nFree; 827 909 } 828 910 #endif /* SQLITE_ENABLE_MEMORY_MANAGEMENT */ 829 911 830 912 #ifdef SQLITE_TEST 831 913 /* ................................................................................ 836 918 int *pnCurrent, /* OUT: Total number of pages cached */ 837 919 int *pnMax, /* OUT: Global maximum cache size */ 838 920 int *pnMin, /* OUT: Sum of PCache1.nMin for purgeable caches */ 839 921 int *pnRecyclable /* OUT: Total number of pages available for recycling */ 840 922 ){ 841 923 PgHdr1 *p; 842 924 int nRecyclable = 0; 843 - for(p=pcache1.pLruHead; p; p=p->pLruNext){ 925 + for(p=pcache1.grp.pLruHead; p; p=p->pLruNext){ 844 926 nRecyclable++; 845 927 } 846 - *pnCurrent = pcache1.nCurrentPage; 847 - *pnMax = pcache1.nMaxPage; 848 - *pnMin = pcache1.nMinPage; 928 + *pnCurrent = pcache1.grp.nCurrentPage; 929 + *pnMax = pcache1.grp.nMaxPage; 930 + *pnMin = pcache1.grp.nMinPage; 849 931 *pnRecyclable = nRecyclable; 850 932 } 851 933 #endif
Changes to src/sqlite.h.in.
711 711 ** chunks (say 1MB at a time), may reduce file-system fragmentation and 712 712 ** improve performance on some systems. 713 713 ** 714 714 ** The [SQLITE_FCNTL_FILE_POINTER] opcode is used to obtain a pointer 715 715 ** to the [sqlite3_file] object associated with a particular database 716 716 ** connection. See the [sqlite3_file_control()] documentation for 717 717 ** additional information. 718 +** 719 +** The [SQLITE_FCNTL_SYNC] opcode is used internally. SQLite calls 720 +** the file-control method with this opcode immediately after the database 721 +** file is synced, or if the database is running in synchronous=off mode 722 +** immediately after it would have been synced otherwise. This makes it 723 +** easier to write special VFS modules that depend on the xSync call. 718 724 */ 719 725 #define SQLITE_FCNTL_LOCKSTATE 1 720 726 #define SQLITE_GET_LOCKPROXYFILE 2 721 727 #define SQLITE_SET_LOCKPROXYFILE 3 722 728 #define SQLITE_LAST_ERRNO 4 723 729 #define SQLITE_FCNTL_SIZE_HINT 5 724 730 #define SQLITE_FCNTL_CHUNK_SIZE 6 725 731 #define SQLITE_FCNTL_FILE_POINTER 7 732 +#define SQLITE_FCNTL_SYNC 8 726 733 727 734 728 735 /* 729 736 ** CAPI3REF: Mutex Handle 730 737 ** 731 738 ** The mutex module within SQLite defines [sqlite3_mutex] to be an 732 739 ** abstract type for a mutex object. The SQLite core never looks ................................................................................ 2300 2307 ** 2301 2308 ** ^(<dt>[SQLITE_OPEN_READWRITE]</dt> 2302 2309 ** <dd>The database is opened for reading and writing if possible, or reading 2303 2310 ** only if the file is write protected by the operating system. In either 2304 2311 ** case the database must already exist, otherwise an error is returned.</dd>)^ 2305 2312 ** 2306 2313 ** ^(<dt>[SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE]</dt> 2307 -** <dd>The database is opened for reading and writing, and is creates it if 2314 +** <dd>The database is opened for reading and writing, and is created if 2308 2315 ** it does not already exist. This is the behavior that is always used for 2309 2316 ** sqlite3_open() and sqlite3_open16().</dd>)^ 2310 2317 ** </dl> 2311 2318 ** 2312 2319 ** If the 3rd parameter to sqlite3_open_v2() is not one of the 2313 2320 ** combinations shown above or one of the combinations shown above combined 2314 2321 ** with the [SQLITE_OPEN_NOMUTEX], [SQLITE_OPEN_FULLMUTEX], ................................................................................ 2650 2657 */ 2651 2658 const char *sqlite3_sql(sqlite3_stmt *pStmt); 2652 2659 2653 2660 /* 2654 2661 ** CAPI3REF: Determine If An SQL Statement Writes The Database 2655 2662 ** 2656 2663 ** ^The sqlite3_stmt_readonly(X) interface returns true (non-zero) if 2657 -** and only if the [prepared statement] X is makes no direct changes to 2664 +** and only if the [prepared statement] X makes no direct changes to 2658 2665 ** the content of the database file. 2659 2666 ** 2660 2667 ** Note that [application-defined SQL functions] or 2661 2668 ** [virtual tables] might change the database indirectly as a side effect. 2662 2669 ** ^(For example, if an application defines a function "eval()" that 2663 2670 ** calls [sqlite3_exec()], then the following SQL statement would 2664 2671 ** change the database file through side-effects: ................................................................................ 3067 3074 ** [SQLITE_MISUSE] means that the this routine was called inappropriately. 3068 3075 ** Perhaps it was called on a [prepared statement] that has 3069 3076 ** already been [sqlite3_finalize | finalized] or on one that had 3070 3077 ** previously returned [SQLITE_ERROR] or [SQLITE_DONE]. Or it could 3071 3078 ** be the case that the same database connection is being used by two or 3072 3079 ** more threads at the same moment in time. 3073 3080 ** 3074 -** For all versions of SQLite up to and including 3.6.23.1, it was required 3075 -** after sqlite3_step() returned anything other than [SQLITE_ROW] that 3076 -** [sqlite3_reset()] be called before any subsequent invocation of 3077 -** sqlite3_step(). Failure to invoke [sqlite3_reset()] in this way would 3078 -** result in an [SQLITE_MISUSE] return from sqlite3_step(). But after 3079 -** version 3.6.23.1, sqlite3_step() began calling [sqlite3_reset()] 3080 -** automatically in this circumstance rather than returning [SQLITE_MISUSE]. 3081 +** For all versions of SQLite up to and including 3.6.23.1, a call to 3082 +** [sqlite3_reset()] was required after sqlite3_step() returned anything 3083 +** other than [SQLITE_ROW] before any subsequent invocation of 3084 +** sqlite3_step(). Failure to reset the prepared statement using 3085 +** [sqlite3_reset()] would result in an [SQLITE_MISUSE] return from 3086 +** sqlite3_step(). But after version 3.6.23.1, sqlite3_step() began 3087 +** calling [sqlite3_reset()] automatically in this circumstance rather 3088 +** than returning [SQLITE_MISUSE]. This is not considered a compatibility 3089 +** break because any application that ever receives an SQLITE_MISUSE error 3090 +** is broken by definition. The [SQLITE_OMIT_AUTORESET] compile-time option 3091 +** can be used to restore the legacy behavior. 3081 3092 ** 3082 3093 ** <b>Goofy Interface Alert:</b> In the legacy interface, the sqlite3_step() 3083 3094 ** API always returns a generic error code, [SQLITE_ERROR], following any 3084 3095 ** error other than [SQLITE_BUSY] and [SQLITE_MISUSE]. You must call 3085 3096 ** [sqlite3_reset()] or [sqlite3_finalize()] in order to find one of the 3086 3097 ** specific [error codes] that better describes the error. 3087 3098 ** We admit that this is a goofy design. The problem has been fixed ................................................................................ 5251 5262 #define SQLITE_MUTEX_RECURSIVE 1 5252 5263 #define SQLITE_MUTEX_STATIC_MASTER 2 5253 5264 #define SQLITE_MUTEX_STATIC_MEM 3 /* sqlite3_malloc() */ 5254 5265 #define SQLITE_MUTEX_STATIC_MEM2 4 /* NOT USED */ 5255 5266 #define SQLITE_MUTEX_STATIC_OPEN 4 /* sqlite3BtreeOpen() */ 5256 5267 #define SQLITE_MUTEX_STATIC_PRNG 5 /* sqlite3_random() */ 5257 5268 #define SQLITE_MUTEX_STATIC_LRU 6 /* lru page list */ 5258 -#define SQLITE_MUTEX_STATIC_LRU2 7 /* lru page list */ 5269 +#define SQLITE_MUTEX_STATIC_LRU2 7 /* NOT USED */ 5270 +#define SQLITE_MUTEX_STATIC_PMEM 7 /* sqlite3PageMalloc() */ 5259 5271 5260 5272 /* 5261 5273 ** CAPI3REF: Retrieve the mutex for a database connection 5262 5274 ** 5263 5275 ** ^This interface returns a pointer the [sqlite3_mutex] object that 5264 5276 ** serializes access to the [database connection] given in the argument 5265 5277 ** when the [threading mode] is Serialized. ................................................................................ 5688 5700 ** 5689 5701 ** ^SQLite invokes the xCreate() method to construct a new cache instance. 5690 5702 ** SQLite will typically create one cache instance for each open database file, 5691 5703 ** though this is not guaranteed. ^The 5692 5704 ** first parameter, szPage, is the size in bytes of the pages that must 5693 5705 ** be allocated by the cache. ^szPage will not be a power of two. ^szPage 5694 5706 ** will the page size of the database file that is to be cached plus an 5695 -** increment (here called "R") of about 100 or 200. SQLite will use the 5707 +** increment (here called "R") of less than 250. SQLite will use the 5696 5708 ** extra R bytes on each page to store metadata about the underlying 5697 5709 ** database page on disk. The value of R depends 5698 5710 ** on the SQLite version, the target platform, and how SQLite was compiled. 5699 -** ^R is constant for a particular build of SQLite. ^The second argument to 5711 +** ^(R is constant for a particular build of SQLite. Except, there are two 5712 +** distinct values of R when SQLite is compiled with the proprietary 5713 +** ZIPVFS extension.)^ ^The second argument to 5700 5714 ** xCreate(), bPurgeable, is true if the cache being created will 5701 5715 ** be used to cache database pages of a file stored on disk, or 5702 5716 ** false if it is used for an in-memory database. The cache implementation 5703 5717 ** does not have to do anything special based with the value of bPurgeable; 5704 5718 ** it is purely advisory. ^On a cache where bPurgeable is false, SQLite will 5705 5719 ** never invoke xUnpin() except to deliberately delete a page. 5706 5720 ** ^In other words, calls to xUnpin() on a cache with bPurgeable set to ................................................................................ 5724 5738 ** 8-byte boundary. The page to be fetched is determined by the key. ^The 5725 5739 ** mimimum key value is 1. After it has been retrieved using xFetch, the page 5726 5740 ** is considered to be "pinned". 5727 5741 ** 5728 5742 ** If the requested page is already in the page cache, then the page cache 5729 5743 ** implementation must return a pointer to the page buffer with its content 5730 5744 ** intact. If the requested page is not already in the cache, then the 5731 -** behavior of the cache implementation should use the value of the createFlag 5745 +** cache implementation should use the value of the createFlag 5732 5746 ** parameter to help it determined what action to take: 5733 5747 ** 5734 5748 ** <table border=1 width=85% align=center> 5735 5749 ** <tr><th> createFlag <th> Behaviour when page is not already in cache 5736 5750 ** <tr><td> 0 <td> Do not allocate a new page. Return NULL. 5737 5751 ** <tr><td> 1 <td> Allocate a new page if it easy and convenient to do so. 5738 5752 ** Otherwise return NULL.
Changes to src/test_config.c.
128 128 #endif 129 129 130 130 #ifdef SQLITE_OMIT_AUTOMATIC_INDEX 131 131 Tcl_SetVar2(interp, "sqlite_options", "autoindex", "0", TCL_GLOBAL_ONLY); 132 132 #else 133 133 Tcl_SetVar2(interp, "sqlite_options", "autoindex", "1", TCL_GLOBAL_ONLY); 134 134 #endif 135 + 136 +#ifdef SQLITE_OMIT_AUTORESET 137 + Tcl_SetVar2(interp, "sqlite_options", "autoreset", "0", TCL_GLOBAL_ONLY); 138 +#else 139 + Tcl_SetVar2(interp, "sqlite_options", "autoreset", "1", TCL_GLOBAL_ONLY); 140 +#endif 135 141 136 142 #ifdef SQLITE_OMIT_AUTOVACUUM 137 143 Tcl_SetVar2(interp, "sqlite_options", "autovacuum", "0", TCL_GLOBAL_ONLY); 138 144 #else 139 145 Tcl_SetVar2(interp, "sqlite_options", "autovacuum", "1", TCL_GLOBAL_ONLY); 140 146 #endif /* SQLITE_OMIT_AUTOVACUUM */ 141 147 #if !defined(SQLITE_DEFAULT_AUTOVACUUM)
Changes to src/test_vfs.c.
536 536 pFd = (TestvfsFd *)ckalloc(sizeof(TestvfsFd) + PARENTVFS(pVfs)->szOsFile); 537 537 memset(pFd, 0, sizeof(TestvfsFd) + PARENTVFS(pVfs)->szOsFile); 538 538 pFd->pShm = 0; 539 539 pFd->pShmId = 0; 540 540 pFd->zFilename = zName; 541 541 pFd->pVfs = pVfs; 542 542 pFd->pReal = (sqlite3_file *)&pFd[1]; 543 + memset(pTestfile, 0, sizeof(TestvfsFile)); 543 544 pTestfile->pFd = pFd; 544 545 545 546 /* Evaluate the Tcl script: 546 547 ** 547 548 ** SCRIPT xOpen FILENAME 548 549 ** 549 550 ** If the script returns an SQLite error code other than SQLITE_OK, an
Changes to src/vdbeInt.h.
35 35 ** The cursor can seek to a BTree entry with a particular key, or 36 36 ** loop over all entries of the Btree. You can also insert new BTree 37 37 ** entries or retrieve the key or data from the entry that the cursor 38 38 ** is currently pointing to. 39 39 ** 40 40 ** Every cursor that the virtual machine has open is represented by an 41 41 ** instance of the following structure. 42 -** 43 -** If the VdbeCursor.isTriggerRow flag is set it means that this cursor is 44 -** really a single row that represents the NEW or OLD pseudo-table of 45 -** a row trigger. The data for the row is stored in VdbeCursor.pData and 46 -** the rowid is in VdbeCursor.iKey. 47 42 */ 48 43 struct VdbeCursor { 49 44 BtCursor *pCursor; /* The cursor structure of the backend */ 45 + Btree *pBt; /* Separate file holding temporary table */ 46 + KeyInfo *pKeyInfo; /* Info about index keys needed by index cursors */ 50 47 int iDb; /* Index of cursor database in db->aDb[] (or -1) */ 51 - i64 lastRowid; /* Last rowid from a Next or NextIdx operation */ 48 + int pseudoTableReg; /* Register holding pseudotable content. */ 49 + int nField; /* Number of fields in the header */ 52 50 Bool zeroed; /* True if zeroed out and ready for reuse */ 53 51 Bool rowidIsValid; /* True if lastRowid is valid */ 54 52 Bool atFirst; /* True if pointing to first entry */ 55 53 Bool useRandomRowid; /* Generate new record numbers semi-randomly */ 56 54 Bool nullRow; /* True if pointing to a row with no data */ 57 55 Bool deferredMoveto; /* A call to sqlite3BtreeMoveto() is needed */ 58 56 Bool isTable; /* True if a table requiring integer keys */ 59 57 Bool isIndex; /* True if an index containing keys only - no data */ 60 58 Bool isOrdered; /* True if the underlying table is BTREE_UNORDERED */ 61 - i64 movetoTarget; /* Argument to the deferred sqlite3BtreeMoveto() */ 62 - Btree *pBt; /* Separate file holding temporary table */ 63 - int pseudoTableReg; /* Register holding pseudotable content. */ 64 - KeyInfo *pKeyInfo; /* Info about index keys needed by index cursors */ 65 - int nField; /* Number of fields in the header */ 66 - i64 seqCount; /* Sequence counter */ 67 59 sqlite3_vtab_cursor *pVtabCursor; /* The cursor for a virtual table */ 68 60 const sqlite3_module *pModule; /* Module for cursor pVtabCursor */ 61 + i64 seqCount; /* Sequence counter */ 62 + i64 movetoTarget; /* Argument to the deferred sqlite3BtreeMoveto() */ 63 + i64 lastRowid; /* Last rowid from a Next or NextIdx operation */ 69 64 70 65 /* Result of last sqlite3BtreeMoveto() done by an OP_NotExists or 71 66 ** OP_IsUnique opcode on this cursor. */ 72 67 int seekResult; 73 68 74 69 /* Cached information about the header for the data record that the 75 70 ** cursor is currently pointing to. Only valid if cacheStatus matches ................................................................................ 133 128 ** A value for VdbeCursor.cacheValid that means the cache is always invalid. 134 129 */ 135 130 #define CACHE_STALE 0 136 131 137 132 /* 138 133 ** Internally, the vdbe manipulates nearly all SQL values as Mem 139 134 ** structures. Each Mem struct may cache multiple representations (string, 140 -** integer etc.) of the same value. A value (and therefore Mem structure) 141 -** has the following properties: 142 -** 143 -** Each value has a manifest type. The manifest type of the value stored 144 -** in a Mem struct is returned by the MemType(Mem*) macro. The type is 145 -** one of SQLITE_NULL, SQLITE_INTEGER, SQLITE_REAL, SQLITE_TEXT or 146 -** SQLITE_BLOB. 135 +** integer etc.) of the same value. 147 136 */ 148 137 struct Mem { 138 + sqlite3 *db; /* The associated database connection */ 139 + char *z; /* String or BLOB value */ 140 + double r; /* Real value */ 149 141 union { 150 - i64 i; /* Integer value. */ 142 + i64 i; /* Integer value used when MEM_Int is set in flags */ 151 143 int nZero; /* Used when bit MEM_Zero is set in flags */ 152 144 FuncDef *pDef; /* Used only when flags==MEM_Agg */ 153 145 RowSet *pRowSet; /* Used only when flags==MEM_RowSet */ 154 146 VdbeFrame *pFrame; /* Used when flags==MEM_Frame */ 155 147 } u; 156 - double r; /* Real value */ 157 - sqlite3 *db; /* The associated database connection */ 158 - char *z; /* String or BLOB value */ 159 148 int n; /* Number of characters in string value, excluding '\0' */ 160 149 u16 flags; /* Some combination of MEM_Null, MEM_Str, MEM_Dyn, etc. */ 161 150 u8 type; /* One of SQLITE_NULL, SQLITE_TEXT, SQLITE_INTEGER, etc */ 162 151 u8 enc; /* SQLITE_UTF8, SQLITE_UTF16BE, SQLITE_UTF16LE */ 163 152 #ifdef SQLITE_DEBUG 164 153 Mem *pScopyFrom; /* This Mem is a shallow copy of pScopyFrom */ 165 154 void *pFiller; /* So that sizeof(Mem) is a multiple of 8 */ ................................................................................ 175 164 ** No other flags may be set in this case. 176 165 ** 177 166 ** If the MEM_Str flag is set then Mem.z points at a string representation. 178 167 ** Usually this is encoded in the same unicode encoding as the main 179 168 ** database (see below for exceptions). If the MEM_Term flag is also 180 169 ** set, then the string is nul terminated. The MEM_Int and MEM_Real 181 170 ** flags may coexist with the MEM_Str flag. 182 -** 183 -** Multiple of these values can appear in Mem.flags. But only one 184 -** at a time can appear in Mem.type. 185 171 */ 186 172 #define MEM_Null 0x0001 /* Value is NULL */ 187 173 #define MEM_Str 0x0002 /* Value is a string */ 188 174 #define MEM_Int 0x0004 /* Value is an integer */ 189 175 #define MEM_Real 0x0008 /* Value is a real number */ 190 176 #define MEM_Blob 0x0010 /* Value is a BLOB */ 191 177 #define MEM_RowSet 0x0020 /* Value is a RowSet object */ ................................................................................ 260 246 VdbeFunc *pVdbeFunc; /* Auxilary data, if created. */ 261 247 Mem s; /* The return value is stored here */ 262 248 Mem *pMem; /* Memory cell used to store aggregate context */ 263 249 int isError; /* Error code returned by the function. */ 264 250 CollSeq *pColl; /* Collating sequence */ 265 251 }; 266 252 267 -/* 268 -** A Set structure is used for quick testing to see if a value 269 -** is part of a small set. Sets are used to implement code like 270 -** this: 271 -** x.y IN ('hi','hoo','hum') 272 -*/ 273 -typedef struct Set Set; 274 -struct Set { 275 - Hash hash; /* A set is just a hash table */ 276 - HashElem *prev; /* Previously accessed hash elemen */ 277 -}; 278 - 279 253 /* 280 254 ** An instance of the virtual machine. This structure contains the complete 281 255 ** state of the virtual machine. 282 256 ** 283 -** The "sqlite3_stmt" structure pointer that is returned by sqlite3_compile() 257 +** The "sqlite3_stmt" structure pointer that is returned by sqlite3_prepare() 284 258 ** is really a pointer to an instance of this structure. 285 259 ** 286 260 ** The Vdbe.inVtabMethod variable is set to non-zero for the duration of 287 261 ** any virtual table method invocations made by the vdbe program. It is 288 262 ** set to 2 for xDestroy method calls and 1 for all other methods. This 289 263 ** variable is used for two purposes: to allow xDestroy methods to execute 290 264 ** "DROP TABLE" statements and to prevent some nasty side effects of 291 265 ** malloc failure when SQLite is invoked recursively by a virtual table 292 266 ** method function. 293 267 */ 294 268 struct Vdbe { 295 269 sqlite3 *db; /* The database connection that owns this statement */ 296 - Vdbe *pPrev,*pNext; /* Linked list of VDBEs with the same Vdbe.db */ 297 - int nOp; /* Number of instructions in the program */ 298 - int nOpAlloc; /* Number of slots allocated for aOp[] */ 299 270 Op *aOp; /* Space to hold the virtual machine's program */ 300 - int nLabel; /* Number of labels used */ 301 - int nLabelAlloc; /* Number of slots allocated in aLabel[] */ 302 - int *aLabel; /* Space to hold the labels */ 271 + Mem *aMem; /* The memory locations */ 303 272 Mem **apArg; /* Arguments to currently executing user function */ 304 273 Mem *aColName; /* Column names to return */ 305 274 Mem *pResultSet; /* Pointer to an array of results */ 275 + int nMem; /* Number of memory locations currently allocated */ 276 + int nOp; /* Number of instructions in the program */ 277 + int nOpAlloc; /* Number of slots allocated for aOp[] */ 278 + int nLabel; /* Number of labels used */ 279 + int nLabelAlloc; /* Number of slots allocated in aLabel[] */ 280 + int *aLabel; /* Space to hold the labels */ 306 281 u16 nResColumn; /* Number of columns in one row of the result set */ 307 282 u16 nCursor; /* Number of slots in apCsr[] */ 283 + u32 magic; /* Magic number for sanity checking */ 284 + char *zErrMsg; /* Error message written here */ 285 + Vdbe *pPrev,*pNext; /* Linked list of VDBEs with the same Vdbe.db */ 308 286 VdbeCursor **apCsr; /* One element of this array for each open cursor */ 309 - u8 errorAction; /* Recovery action to do in case of an error */ 310 - u8 okVar; /* True if azVar[] has been initialized */ 311 - ynVar nVar; /* Number of entries in aVar[] */ 312 287 Mem *aVar; /* Values for the OP_Variable opcode. */ 313 288 char **azVar; /* Name of variables */ 314 - u32 magic; /* Magic number for sanity checking */ 315 - int nMem; /* Number of memory locations currently allocated */ 316 - Mem *aMem; /* The memory locations */ 289 + ynVar nVar; /* Number of entries in aVar[] */ 317 290 u32 cacheCtr; /* VdbeCursor row cache generation counter */ 318 291 int pc; /* The program counter */ 319 292 int rc; /* Value to return */ 320 - char *zErrMsg; /* Error message written here */ 293 + u8 errorAction; /* Recovery action to do in case of an error */ 294 + u8 okVar; /* True if azVar[] has been initialized */ 321 295 u8 explain; /* True if EXPLAIN present on SQL command */ 322 296 u8 changeCntOn; /* True to update the change-counter */ 323 297 u8 expired; /* True if the VM needs to be recompiled */ 324 298 u8 runOnlyOnce; /* Automatically expire on reset */ 325 299 u8 minWriteFileFormat; /* Minimum file format for writable database files */ 326 300 u8 inVtabMethod; /* See comments above */ 327 301 u8 usesStmtJournal; /* True if uses a statement journal */ 328 302 u8 readOnly; /* True for read-only statements */ 329 303 u8 isPrepareV2; /* True if prepared with prepare_v2() */ 330 304 int nChange; /* Number of db changes made since last reset */ 331 305 int btreeMask; /* Bitmask of db->aDb[] entries referenced */ 332 - i64 startTime; /* Time when query started - used for profiling */ 333 - BtreeMutexArray aMutex; /* An array of Btree used here and needing locks */ 306 + int iStatement; /* Statement number (or 0 if has not opened stmt) */ 334 307 int aCounter[3]; /* Counters used by sqlite3_stmt_status() */ 335 - char *zSql; /* Text of the SQL statement that generated this */ 336 - void *pFree; /* Free this when deleting the vdbe */ 308 + BtreeMutexArray aMutex; /* An array of Btree used here and needing locks */ 309 +#ifndef SQLITE_OMIT_TRACE 310 + i64 startTime; /* Time when query started - used for profiling */ 311 +#endif 337 312 i64 nFkConstraint; /* Number of imm. FK constraints this VM */ 338 313 i64 nStmtDefCons; /* Number of def. constraints when stmt started */ 339 - int iStatement; /* Statement number (or 0 if has not opened stmt) */ 314 + char *zSql; /* Text of the SQL statement that generated this */ 315 + void *pFree; /* Free this when deleting the vdbe */ 340 316 #ifdef SQLITE_DEBUG 341 317 FILE *trace; /* Write an execution trace here, if not NULL */ 342 318 #endif 343 319 VdbeFrame *pFrame; /* Parent frame */ 344 320 VdbeFrame *pDelFrame; /* List of frame objects to free on VM reset */ 345 321 int nFrame; /* Number of frames in pFrame list */ 346 322 u32 expmask; /* Binding to these vars invalidates VM */
Changes to src/vdbeapi.c.
341 341 static int sqlite3Step(Vdbe *p){ 342 342 sqlite3 *db; 343 343 int rc; 344 344 345 345 assert(p); 346 346 if( p->magic!=VDBE_MAGIC_RUN ){ 347 347 /* We used to require that sqlite3_reset() be called before retrying 348 - ** sqlite3_step() after any error. But after 3.6.23, we changed this 349 - ** so that sqlite3_reset() would be called automatically instead of 350 - ** throwing the error. 348 + ** sqlite3_step() after any error or after SQLITE_DONE. But beginning 349 + ** with version 3.7.0, we changed this so that sqlite3_reset() would 350 + ** be called automatically instead of throwing the SQLITE_MISUSE error. 351 + ** This "automatic-reset" change is not technically an incompatibility, 352 + ** since any application that receives an SQLITE_MISUSE is broken by 353 + ** definition. 354 + ** 355 + ** Nevertheless, some published applications that were originally written 356 + ** for version 3.6.23 or earlier do in fact depend on SQLITE_MISUSE 357 + ** returns, and the so were broken by the automatic-reset change. As a 358 + ** a work-around, the SQLITE_OMIT_AUTORESET compile-time restores the 359 + ** legacy behavior of returning SQLITE_MISUSE for cases where the 360 + ** previous sqlite3_step() returned something other than a SQLITE_LOCKED 361 + ** or SQLITE_BUSY error. 351 362 */ 363 +#ifdef SQLITE_OMIT_AUTORESET 364 + if( p->rc==SQLITE_BUSY || p->rc==SQLITE_LOCKED ){ 352 365 sqlite3_reset((sqlite3_stmt*)p); 366 + }else{ 367 + return SQLITE_MISUSE_BKPT; 368 + } 369 +#else 370 + sqlite3_reset((sqlite3_stmt*)p); 371 +#endif 353 372 } 354 373 355 374 /* Check that malloc() has not failed. If it has, return early. */ 356 375 db = p->db; 357 376 if( db->mallocFailed ){ 358 377 p->rc = SQLITE_NOMEM; 359 378 return SQLITE_NOMEM; ................................................................................ 681 700 ** this assert() from failing, when building with SQLITE_DEBUG defined 682 701 ** using gcc, force nullMem to be 8-byte aligned using the magical 683 702 ** __attribute__((aligned(8))) macro. */ 684 703 static const Mem nullMem 685 704 #if defined(SQLITE_DEBUG) && defined(__GNUC__) 686 705 __attribute__((aligned(8))) 687 706 #endif 688 - = {{0}, (double)0, 0, "", 0, MEM_Null, SQLITE_NULL, 0, 0, 0 }; 707 + = {0, "", (double)0, {0}, 0, MEM_Null, SQLITE_NULL, 0, 0, 0 }; 689 708 690 709 if( pVm && ALWAYS(pVm->db) ){ 691 710 sqlite3_mutex_enter(pVm->db->mutex); 692 711 sqlite3Error(pVm->db, SQLITE_RANGE, 0); 693 712 } 694 713 pOut = (Mem*)&nullMem; 695 714 }
Changes to test/capi2.test.
70 70 } {SQLITE_DONE} 71 71 do_test capi2-1.7 { 72 72 list [sqlite3_column_count $VM] [get_row_values $VM] [get_column_names $VM] 73 73 } {2 {} {name rowid text INTEGER}} 74 74 75 75 # This used to be SQLITE_MISUSE. But now we automatically reset prepared 76 76 # statements. 77 +ifcapable autoreset { 77 78 do_test capi2-1.8 { 78 79 sqlite3_step $VM 79 80 } {SQLITE_ROW} 81 +} else { 82 + do_test capi2-1.8 { 83 + sqlite3_step $VM 84 + } {SQLITE_MISUSE} 85 +} 80 86 81 87 # Update: In v2, once SQLITE_MISUSE is returned the statement handle cannot 82 88 # be interrogated for more information. However in v3, since the column 83 89 # count, names and types are determined at compile time, these are still 84 90 # accessible after an SQLITE_MISUSE error. 85 91 do_test capi2-1.9 { 86 92 sqlite3_reset $VM
Changes to test/exclusive2.test.
295 295 } {4} 296 296 do_test exclusive2-3.5 { 297 297 execsql { 298 298 PRAGMA locking_mode = normal; 299 299 INSERT INTO t1 VALUES(randstr(10, 400)); 300 300 } 301 301 readPagerChangeCounter test.db 302 -} {4} 302 +} {5} 303 303 do_test exclusive2-3.6 { 304 304 execsql { 305 305 INSERT INTO t1 VALUES(randstr(10, 400)); 306 306 } 307 307 readPagerChangeCounter test.db 308 -} {5} 308 +} {6} 309 309 sqlite3_soft_heap_limit $cmdlinearg(soft-heap-limit) 310 310 311 311 finish_test
Changes to test/fkey2.test.
1410 1410 INSERT INTO one VALUES(1, 2, 3); 1411 1411 } 1412 1412 } {1} 1413 1413 do_test fkey2-17.1.2 { 1414 1414 set STMT [sqlite3_prepare_v2 db "INSERT INTO two VALUES(4, 5, 6)" -1 dummy] 1415 1415 sqlite3_step $STMT 1416 1416 } {SQLITE_CONSTRAINT} 1417 +ifcapable autoreset { 1417 1418 do_test fkey2-17.1.3 { 1418 1419 sqlite3_step $STMT 1419 1420 } {SQLITE_CONSTRAINT} 1421 +} else { 1422 + do_test fkey2-17.1.3 { 1423 + sqlite3_step $STMT 1424 + } {SQLITE_MISUSE} 1425 +} 1420 1426 do_test fkey2-17.1.4 { 1421 1427 sqlite3_finalize $STMT 1422 1428 } {SQLITE_CONSTRAINT} 1423 1429 do_test fkey2-17.1.5 { 1424 1430 execsql { 1425 1431 INSERT INTO one VALUES(2, 3, 4); 1426 1432 INSERT INTO one VALUES(3, 4, 5);
Changes to test/fts3matchinfo.test.
333 333 SELECT typeof(matchinfo(t10)), length(matchinfo(t10)) FROM t10 WHERE docid=1; 334 334 } {blob 0} 335 335 do_execsql_test 7.4 { 336 336 SELECT typeof(matchinfo(t10)), length(matchinfo(t10)) 337 337 FROM t10 WHERE t10 MATCH 'record' 338 338 } {blob 20 blob 20} 339 339 340 +#------------------------------------------------------------------------- 341 +# Test a special case - matchinfo('nxa') with many zero length documents. 342 +# Special because "x" internally uses a statement used by both "n" and "a". 343 +# This was causing a problem at one point in the obscure case where the 344 +# total number of bytes of data stored in an fts3 table was greater than 345 +# the number of rows. i.e. when the following query returns true: 346 +# 347 +# SELECT sum(length(content)) < count(*) FROM fts4table; 348 +# 349 +do_execsql_test 8.1 { 350 + CREATE VIRTUAL TABLE t11 USING fts4; 351 + INSERT INTO t11(t11) VALUES('nodesize=24'); 352 + INSERT INTO t11 VALUES('quitealongstringoftext'); 353 + INSERT INTO t11 VALUES('anotherquitealongstringoftext'); 354 + INSERT INTO t11 VALUES('athirdlongstringoftext'); 355 + INSERT INTO t11 VALUES('andonemoreforgoodluck'); 356 +} 357 +do_test 8.2 { 358 + for {set i 0} {$i < 200} {incr i} { 359 + execsql { INSERT INTO t11 VALUES('') } 360 + } 361 + execsql { INSERT INTO t11(t11) VALUES('optimize') } 362 +} {} 363 +do_execsql_test 8.3 { 364 + SELECT mit(matchinfo(t11, 'nxa')) FROM t11 WHERE t11 MATCH 'a*' 365 +} {{204 1 3 3 0} {204 1 3 3 0} {204 1 3 3 0}} 366 + 367 +# Corruption related tests. 368 +do_execsql_test 8.4.1.1 { UPDATE t11_stat SET value = X'0000'; } 369 +do_catchsql_test 8.5.1.2 { 370 + SELECT mit(matchinfo(t11, 'nxa')) FROM t11 WHERE t11 MATCH 'a*' 371 +} {1 {database disk image is malformed}} 372 + 373 +do_execsql_test 8.4.2.1 { UPDATE t11_stat SET value = X'00'; } 374 +do_catchsql_test 8.5.2.2 { 375 + SELECT mit(matchinfo(t11, 'nxa')) FROM t11 WHERE t11 MATCH 'a*' 376 +} {1 {database disk image is malformed}} 377 + 378 +do_execsql_test 8.4.3.1 { UPDATE t11_stat SET value = NULL; } 379 +do_catchsql_test 8.5.3.2 { 380 + SELECT mit(matchinfo(t11, 'nxa')) FROM t11 WHERE t11 MATCH 'a*' 381 +} {1 {database disk image is malformed}} 340 382 341 383 finish_test 342 384
Changes to test/jrnlmode3.test.
41 41 do_test jrnlmode3-1.2 { 42 42 db eval { 43 43 BEGIN; 44 44 INSERT INTO t1 VALUES(2); 45 45 ROLLBACK; 46 46 SELECT * FROM t1; 47 47 } 48 -} {1 2} 48 +} {1} 49 49 50 50 db close 51 51 file delete -force test.db test.db-journal 52 52 sqlite3 db test.db 53 53 54 54 do_test jrnlmode3-2.1 { 55 55 db eval { ................................................................................ 63 63 do_test jrnlmode3-2.2 { 64 64 db eval { 65 65 BEGIN; 66 66 INSERT INTO t1 VALUES(2); 67 67 ROLLBACK; 68 68 SELECT * FROM t1; 69 69 } 70 -} {1 2} 70 +} {1} 71 71 72 72 # Test cases to verify that we can move from any journal_mode 73 73 # to any other, as long as we are not in a transaction. Verify 74 74 # that we cannot change journal_mode while a transaction is active. 75 75 # 76 76 set all_journal_modes {delete persist truncate memory off} 77 77 set cnt 0 ................................................................................ 108 108 CREATE TABLE t1(x); 109 109 BEGIN; 110 110 INSERT INTO t1 VALUES($cnt); 111 111 } 112 112 db eval "PRAGMA journal_mode=$tojmode" 113 113 } $fromjmode 114 114 115 - # Rollback the transaction. Verify that the rollback occurred 116 - # if journal_mode!=OFF. 115 + # Rollback the transaction. 117 116 # 118 117 do_test jrnlmode3-3.$cnt.4 { 119 118 db eval { 120 119 ROLLBACK; 121 120 SELECT * FROM t1; 122 121 } 123 - } [expr {$fromjmode=="off"?$cnt:""}] 122 + } {} 124 123 125 124 # Now change the journal mode again. This time the new mode 126 125 # should take. 127 126 # 128 127 do_test jrnlmode3-3.$cnt.5 { 129 128 db eval "PRAGMA journal_mode=$tojmode" 130 129 } $tojmode ................................................................................ 139 138 BEGIN; 140 139 INSERT INTO t1 VALUES(1); 141 140 } 142 141 db eval ROLLBACK 143 142 db eval { 144 143 SELECT * FROM t1; 145 144 } 146 - } [expr {$tojmode=="off"?"1":""}] 145 + } {} 147 146 } 148 147 } 149 148 150 149 finish_test
Changes to test/memsubsys1.test.
92 92 db close 93 93 sqlite3_shutdown 94 94 sqlite3_config_pagecache [expr 1024+$xtra_size] 20 95 95 sqlite3_initialize 96 96 reset_highwater_marks 97 97 build_test_db memsubsys1-2 {PRAGMA page_size=1024} 98 98 #show_memstats 99 +set MEMORY_MANAGEMENT $sqlite_options(memorymanage) 99 100 do_test memsubsys1-2.3 { 100 101 set pg_ovfl [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_OVERFLOW 0] 2] 101 -} [expr ($AUTOVACUUM+$TEMP_STORE>=2)*1024] 102 +} [expr ($AUTOVACUUM+$TEMP_STORE+$MEMORY_MANAGEMENT>=3)*1024] 102 103 do_test memsubsys1-2.4 { 103 104 set pg_used [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_USED 0] 2] 104 105 } 20 105 106 do_test memsubsys1-2.5 { 106 107 set s_used [lindex [sqlite3_status SQLITE_STATUS_SCRATCH_USED 0] 2] 107 108 } 0 108 109
Changes to test/mutex1.test.
116 116 clear_mutex_counters 117 117 sqlite3 db test.db -nomutex 0 -fullmutex 0 118 118 catchsql { CREATE TABLE abc(a, b, c) } 119 119 db eval { 120 120 INSERT INTO abc VALUES(1, 2, 3); 121 121 } 122 122 } {} 123 - 123 + ifcapable !memorymanage { 124 + regsub { static_lru} $mutexes {} mutexes 125 + } 124 126 do_test mutex1.2.$mode.3 { 125 127 mutex_counters counters 126 128 127 129 set res [list] 128 130 foreach {key value} [array get counters] { 129 131 if {$key ne "total" && $value > 0} { 130 132 lappend res $key
Changes to test/pager1.test.
1610 1610 do_catchsql_test pager1-14.1.2 { 1611 1611 BEGIN; 1612 1612 INSERT INTO t1 VALUES(3, 4); 1613 1613 ROLLBACK; 1614 1614 } {0 {}} 1615 1615 do_execsql_test pager1-14.1.3 { 1616 1616 SELECT * FROM t1; 1617 -} {1 2 3 4} 1617 +} {1 2} 1618 1618 do_catchsql_test pager1-14.1.4 { 1619 1619 BEGIN; 1620 1620 INSERT INTO t1(rowid, a, b) SELECT a+3, b, b FROM t1; 1621 1621 INSERT INTO t1(rowid, a, b) SELECT a+3, b, b FROM t1; 1622 1622 } {1 {PRIMARY KEY must be unique}} 1623 1623 do_execsql_test pager1-14.1.5 { 1624 1624 COMMIT; 1625 1625 SELECT * FROM t1; 1626 -} {1 2 3 4 2 2 4 4} 1626 +} {1 2 2 2} 1627 1627 1628 1628 #------------------------------------------------------------------------- 1629 1629 # Test opening and closing the pager sub-system with different values 1630 1630 # for the sqlite3_vfs.szOsFile variable. 1631 1631 # 1632 1632 faultsim_delete_and_reopen 1633 1633 do_execsql_test pager1-15.0 {
Changes to test/pager2.test.
129 129 CREATE TABLE t1(a, b); 130 130 PRAGMA journal_mode = off; 131 131 BEGIN; 132 132 INSERT INTO t1 VALUES(1, 2); 133 133 ROLLBACK; 134 134 SELECT * FROM t1; 135 135 } 136 -} {off 1 2} 136 +} {off} 137 137 do_test pager2-2.2 { 138 138 faultsim_delete_and_reopen 139 139 execsql { 140 140 PRAGMA auto_vacuum = incremental; 141 141 PRAGMA page_size = 1024; 142 142 PRAGMA journal_mode = off; 143 143 CREATE TABLE t1(a, b);
Changes to test/pcache.test.
16 16 set testdir [file dirname $argv0] 17 17 source $testdir/tester.tcl 18 18 19 19 # Do not use a codec for tests in this file, as the database file is 20 20 # manipulated directly using tcl scripts (using the [hexio_write] command). 21 21 # 22 22 do_not_use_codec 23 + 24 +# Only works with a mode-2 pcache where all pcaches share a single set 25 +# of pages. 26 +# 27 +ifcapable {!memorymanage && threadsafe} { 28 + finish_test 29 + return 30 +} 23 31 24 32 # The pcache module limits the number of pages available to purgeable 25 33 # caches to the sum of the 'cache_size' values for the set of open 26 34 # caches. This block of tests, pcache-1.*, test that the library behaves 27 35 # corrctly when it is forced to exceed this limit. 28 36 # 29 37 do_test pcache-1.1 {
test/progress.test became a regular file.
whitespace changes only
Changes to test/savepoint.test.
901 901 INSERT INTO t1 VALUES(13, 14); 902 902 SAVEPOINT s1; 903 903 INSERT INTO t1 VALUES(15, 16); 904 904 ROLLBACK TO s1; 905 905 ROLLBACK; 906 906 SELECT * FROM t1; 907 907 } 908 - } {1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16} 908 + } {1 2 3 4 5 6 7 8 9 10 11 12} 909 909 } 910 910 911 911 db close 912 912 file delete test.db 913 913 do_multiclient_test tn { 914 914 do_test savepoint-14.$tn.1 { 915 915 sql1 {
Added test/tkt-5d863f876e.test.
1 +# 2011 January 15 2 +# 3 +# The author disclaims copyright to this source code. In place of 4 +# a legal notice, here is a blessing: 5 +# 6 +# May you do good and not evil. 7 +# May you find forgiveness for yourself and forgive others. 8 +# May you share freely, never taking more than you give. 9 +# 10 +#*********************************************************************** 11 +# This file implements regression tests for SQLite library. 12 +# 13 +# This file implements tests to verify that ticket [5d863f876e] has been 14 +# fixed. 15 +# 16 + 17 +set testdir [file dirname $argv0] 18 +source $testdir/tester.tcl 19 +source $testdir/lock_common.tcl 20 + 21 +do_multiclient_test tn { 22 + do_test $tn.1 { 23 + sql1 { 24 + CREATE TABLE t1(a, b); 25 + CREATE INDEX i1 ON t1(a, b); 26 + INSERT INTO t1 VALUES(1, 2); 27 + INSERT INTO t1 VALUES(3, 4); 28 + PRAGMA journal_mode = WAL; 29 + VACUUM; 30 + PRAGMA journal_mode = DELETE; 31 + } 32 + } {wal delete} 33 + 34 + do_test $tn.2 { 35 + sql2 { SELECT * FROM t1 } 36 + } {1 2 3 4} 37 + 38 + do_test $tn.3 { 39 + sql1 { 40 + INSERT INTO t1 VALUES(5, 6); 41 + PRAGMA journal_mode = WAL; 42 + VACUUM; 43 + PRAGMA journal_mode = DELETE; 44 + } 45 + } {wal delete} 46 + 47 + do_test $tn.2 { 48 + sql2 { PRAGMA integrity_check } 49 + } {ok} 50 +} 51 + 52 + 53 +finish_test
tool/mkopts.tcl became a regular file.
whitespace changes only
Changes to tool/mksqlite3h.tcl.
49 49 50 50 # Get the fossil-scm check-in date from the "D" card of $TOP/manifest. 51 51 # 52 52 set in [open $TOP/manifest] 53 53 set zDate {} 54 54 while {![eof $in]} { 55 55 set line [gets $in] 56 - if {[regexp {^D (2.*[0-9])} $line all date]} { 56 + if {[regexp {^D (2[-0-9T:]+)} $line all date]} { 57 57 set zDate [string map {T { }} $date] 58 58 break 59 59 } 60 60 } 61 61 close $in 62 62 63 63 # Set up patterns for recognizing API declarations.
Changes to tool/shell1.test.
196 196 list $rc \ 197 197 [regexp {Error: missing argument for option: -nullvalue} $res] 198 198 } {1 1} 199 199 200 200 # -version show SQLite version 201 201 do_test shell1-1.16.1 { 202 202 catchcmd "-version test.db" "" 203 -} {0 3.7.3} 203 +} {0 3.7.5} 204 204 205 205 #---------------------------------------------------------------------------- 206 206 # Test cases shell1-2.*: Basic "dot" command token parsing. 207 207 # 208 208 209 209 # check first token handling 210 210 do_test shell1-2.1.1 {
Changes to tool/showdb.c.
53 53 ** Read content from the file. 54 54 ** 55 55 ** Space to hold the content is obtained from malloc() and needs to be 56 56 ** freed by the caller. 57 57 */ 58 58 static unsigned char *getContent(int ofst, int nByte){ 59 59 unsigned char *aData; 60 - aData = malloc(nByte); 60 + aData = malloc(nByte+32); 61 61 if( aData==0 ) out_of_memory(); 62 + memset(aData, 0, nByte+32); 62 63 lseek(db, ofst, SEEK_SET); 63 64 read(db, aData, nByte); 64 65 return aData; 65 66 } 66 67 67 68 /* 68 69 ** Print a range of bytes as hex and as ascii. ................................................................................ 176 177 print_decode_line(aData, 80, 4, "meta[10]"); 177 178 print_decode_line(aData, 84, 4, "meta[11]"); 178 179 print_decode_line(aData, 88, 4, "meta[12]"); 179 180 print_decode_line(aData, 92, 4, "Change counter for version number"); 180 181 print_decode_line(aData, 96, 4, "SQLite version number"); 181 182 } 182 183 184 +/* 185 +** Describe cell content. 186 +*/ 187 +static int describeContent( 188 + unsigned char *a, /* Cell content */ 189 + int nLocal, /* Bytes in a[] */ 190 + char *zDesc /* Write description here */ 191 +){ 192 + int nDesc = 0; 193 + int n, i, j; 194 + i64 x, v; 195 + const unsigned char *pData; 196 + const unsigned char *pLimit; 197 + char sep = ' '; 198 + 199 + pLimit = &a[nLocal]; 200 + n = decodeVarint(a, &x); 201 + pData = &a[x]; 202 + a += n; 203 + i = x - n; 204 + while( i>0 && pData<=pLimit ){ 205 + n = decodeVarint(a, &x); 206 + a += n; 207 + i -= n; 208 + nLocal -= n; 209 + zDesc[0] = sep; 210 + sep = ','; 211 + nDesc++; 212 + zDesc++; 213 + if( x==0 ){ 214 + sprintf(zDesc, "*"); /* NULL is a "*" */ 215 + }else if( x>=1 && x<=6 ){ 216 + v = (signed char)pData[0]; 217 + pData++; 218 + switch( x ){ 219 + case 6: v = (v<<16) + (pData[0]<<8) + pData[1]; pData += 2; 220 + case 5: v = (v<<16) + (pData[0]<<8) + pData[1]; pData += 2; 221 + case 4: v = (v<<8) + pData[0]; pData++; 222 + case 3: v = (v<<8) + pData[0]; pData++; 223 + case 2: v = (v<<8) + pData[0]; pData++; 224 + } 225 + sprintf(zDesc, "%lld", v); 226 + }else if( x==7 ){ 227 + sprintf(zDesc, "real"); 228 + pData += 8; 229 + }else if( x==8 ){ 230 + sprintf(zDesc, "0"); 231 + }else if( x==9 ){ 232 + sprintf(zDesc, "1"); 233 + }else if( x>=12 ){ 234 + int size = (x-12)/2; 235 + if( (x&1)==0 ){ 236 + sprintf(zDesc, "blob(%d)", size); 237 + }else{ 238 + sprintf(zDesc, "txt(%d)", size); 239 + } 240 + pData += size; 241 + } 242 + j = strlen(zDesc); 243 + zDesc += j; 244 + nDesc += j; 245 + } 246 + return nDesc; 247 +} 248 + 249 +/* 250 +** Compute the local payload size given the total payload size and 251 +** the page size. 252 +*/ 253 +static int localPayload(i64 nPayload, char cType){ 254 + int maxLocal; 255 + int minLocal; 256 + int surplus; 257 + int nLocal; 258 + if( cType==13 ){ 259 + /* Table leaf */ 260 + maxLocal = pagesize-35; 261 + minLocal = (pagesize-12)*32/255-23; 262 + }else{ 263 + maxLocal = (pagesize-12)*64/255-23; 264 + minLocal = (pagesize-12)*32/255-23; 265 + } 266 + if( nPayload>maxLocal ){ 267 + surplus = minLocal + (nPayload-minLocal)%(pagesize-4); 268 + if( surplus<=maxLocal ){ 269 + nLocal = surplus; 270 + }else{ 271 + nLocal = minLocal; 272 + } 273 + }else{ 274 + nLocal = nPayload; 275 + } 276 + return nLocal; 277 +} 278 + 279 + 183 280 /* 184 281 ** Create a description for a single cell. 282 +** 283 +** The return value is the local cell size. 185 284 */ 186 -static int describeCell(unsigned char cType, unsigned char *a, char **pzDesc){ 285 +static int describeCell( 286 + unsigned char cType, /* Page type */ 287 + unsigned char *a, /* Cell content */ 288 + int showCellContent, /* Show cell content if true */ 289 + char **pzDesc /* Store description here */ 290 +){ 187 291 int i; 188 292 int nDesc = 0; 189 293 int n = 0; 190 294 int leftChild; 191 295 i64 nPayload; 192 296 i64 rowid; 193 - static char zDesc[100]; 297 + int nLocal; 298 + static char zDesc[1000]; 194 299 i = 0; 195 300 if( cType<=5 ){ 196 301 leftChild = ((a[0]*256 + a[1])*256 + a[2])*256 + a[3]; 197 302 a += 4; 198 303 n += 4; 199 - sprintf(zDesc, "left-child: %d ", leftChild); 304 + sprintf(zDesc, "lx: %d ", leftChild); 200 305 nDesc = strlen(zDesc); 201 306 } 202 307 if( cType!=5 ){ 203 308 i = decodeVarint(a, &nPayload); 204 309 a += i; 205 310 n += i; 206 - sprintf(&zDesc[nDesc], "sz: %lld ", nPayload); 311 + sprintf(&zDesc[nDesc], "n: %lld ", nPayload); 207 312 nDesc += strlen(&zDesc[nDesc]); 313 + nLocal = localPayload(nPayload, cType); 314 + }else{ 315 + nPayload = nLocal = 0; 208 316 } 209 317 if( cType==5 || cType==13 ){ 210 318 i = decodeVarint(a, &rowid); 211 319 a += i; 212 320 n += i; 213 - sprintf(&zDesc[nDesc], "rowid: %lld ", rowid); 321 + sprintf(&zDesc[nDesc], "r: %lld ", rowid); 322 + nDesc += strlen(&zDesc[nDesc]); 323 + } 324 + if( nLocal<nPayload ){ 325 + int ovfl; 326 + unsigned char *b = &a[nLocal]; 327 + ovfl = ((b[0]*256 + b[1])*256 + b[2])*256 + b[3]; 328 + sprintf(&zDesc[nDesc], "ov: %d ", ovfl); 214 329 nDesc += strlen(&zDesc[nDesc]); 330 + n += 4; 331 + } 332 + if( showCellContent && cType!=5 ){ 333 + nDesc += describeContent(a, nLocal, &zDesc[nDesc-1]); 215 334 } 216 335 *pzDesc = zDesc; 217 - return n; 336 + return nLocal+n; 218 337 } 219 338 220 339 /* 221 340 ** Decode a btree page 222 341 */ 223 -static void decode_btree_page(unsigned char *a, int pgno, int hdrSize){ 342 +static void decode_btree_page( 343 + unsigned char *a, /* Page content */ 344 + int pgno, /* Page number */ 345 + int hdrSize, /* Size of the page header. 0 or 100 */ 346 + char *zArgs /* Flags to control formatting */ 347 +){ 224 348 const char *zType = "unknown"; 225 349 int nCell; 226 - int i; 350 + int i, j; 227 351 int iCellPtr; 352 + int showCellContent = 0; 353 + int showMap = 0; 354 + char *zMap = 0; 228 355 switch( a[0] ){ 229 356 case 2: zType = "index interior node"; break; 230 357 case 5: zType = "table interior node"; break; 231 358 case 10: zType = "index leaf"; break; 232 359 case 13: zType = "table leaf"; break; 360 + } 361 + while( zArgs[0] ){ 362 + switch( zArgs[0] ){ 363 + case 'c': showCellContent = 1; break; 364 + case 'm': showMap = 1; break; 365 + } 366 + zArgs++; 233 367 } 234 368 printf("Decode of btree page %d:\n", pgno); 235 369 print_decode_line(a, 0, 1, zType); 236 370 print_decode_line(a, 1, 2, "Offset to first freeblock"); 237 371 print_decode_line(a, 3, 2, "Number of cells on this page"); 238 372 nCell = a[3]*256 + a[4]; 239 373 print_decode_line(a, 5, 2, "Offset to cell content area"); ................................................................................ 240 374 print_decode_line(a, 7, 1, "Fragmented byte count"); 241 375 if( a[0]==2 || a[0]==5 ){ 242 376 print_decode_line(a, 8, 4, "Right child"); 243 377 iCellPtr = 12; 244 378 }else{ 245 379 iCellPtr = 8; 246 380 } 381 + if( nCell>0 ){ 382 + printf(" key: lx=left-child n=payload-size r=rowid\n"); 383 + } 384 + if( showMap ){ 385 + zMap = malloc(pagesize); 386 + memset(zMap, '.', pagesize); 387 + memset(zMap, '1', hdrSize); 388 + memset(&zMap[hdrSize], 'H', iCellPtr); 389 + memset(&zMap[hdrSize+iCellPtr], 'P', 2*nCell); 390 + } 247 391 for(i=0; i<nCell; i++){ 248 392 int cofst = iCellPtr + i*2; 249 393 char *zDesc; 394 + int n; 395 + 250 396 cofst = a[cofst]*256 + a[cofst+1]; 251 - describeCell(a[0], &a[cofst-hdrSize], &zDesc); 397 + n = describeCell(a[0], &a[cofst-hdrSize], showCellContent, &zDesc); 398 + if( showMap ){ 399 + char zBuf[30]; 400 + memset(&zMap[cofst], '*', n); 401 + zMap[cofst] = '['; 402 + zMap[cofst+n-1] = ']'; 403 + sprintf(zBuf, "%d", i); 404 + j = strlen(zBuf); 405 + if( j<=n-2 ) memcpy(&zMap[cofst+1], zBuf, j); 406 + } 252 407 printf(" %03x: cell[%d] %s\n", cofst, i, zDesc); 408 + } 409 + if( showMap ){ 410 + for(i=0; i<pagesize; i+=64){ 411 + printf(" %03x: %.64s\n", i, &zMap[i]); 412 + } 413 + free(zMap); 253 414 } 254 415 } 255 416 256 417 /* 257 418 ** Decode a freelist trunk page. 258 419 */ 259 420 static void decode_trunk_page( ................................................................................ 296 457 fprintf(stderr, "Usage %s FILENAME ?args...?\n\n", argv0); 297 458 fprintf(stderr, 298 459 "args:\n" 299 460 " dbheader Show database header\n" 300 461 " NNN..MMM Show hex of pages NNN through MMM\n" 301 462 " NNN..end Show hex of pages NNN through end of file\n" 302 463 " NNNb Decode btree page NNN\n" 464 + " NNNbc Decode btree page NNN and show content\n" 465 + " NNNbm Decode btree page NNN and show a layout map\n" 303 466 " NNNt Decode freelist trunk page NNN\n" 304 467 " NNNtd Show leave freelist pages on the decode\n" 305 468 " NNNtr Recurisvely decode freelist starting at NNN\n" 306 469 ); 307 470 } 308 471 309 472 int main(int argc, char **argv){ ................................................................................ 357 520 nByte = pagesize-100; 358 521 }else{ 359 522 hdrSize = 0; 360 523 ofst = (iStart-1)*pagesize; 361 524 nByte = pagesize; 362 525 } 363 526 a = getContent(ofst, nByte); 364 - decode_btree_page(a, iStart, hdrSize); 527 + decode_btree_page(a, iStart, hdrSize, &zLeft[1]); 365 528 free(a); 366 529 continue; 367 530 }else if( zLeft && zLeft[0]=='t' ){ 368 531 unsigned char *a; 369 532 int detail = 0; 370 533 int recursive = 0; 371 534 int i;