Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Merge latest trunk changes into this branch. |
|---|---|
| Downloads: | Tarball | ZIP archive | SQL archive |
| Timelines: | family | ancestors | descendants | both | sqlar-shell-support |
| Files: | files | file ages | folders |
| SHA3-256: |
803156cba8b056a1cb8d1bb186a57454 |
| User & Date: | dan 2017-12-14 13:55:01 |
Context
|
2017-12-14
| ||
| 15:40 | Improve error and usage messages output by the shell ".ar" command. check-in: b9d2d5d9 user: dan tags: sqlar-shell-support | |
| 13:55 | Merge latest trunk changes into this branch. check-in: 803156cb user: dan tags: sqlar-shell-support | |
|
2017-12-13
| ||
| 23:47 | In valueFromExpr() only generate a OOM fault if there have been no prior faults. check-in: 3765aaf7 user: drh tags: trunk | |
| 20:17 | Add the shell tool ".ar --update" command. check-in: 825e3c03 user: dan tags: sqlar-shell-support | |
Changes
Changes to Makefile.msc.
2157 2157 fts5parse.c fts5parse.h \ 2158 2158 $(TOP)\ext\fts5\fts5_storage.c \ 2159 2159 $(TOP)\ext\fts5\fts5_tokenize.c \ 2160 2160 $(TOP)\ext\fts5\fts5_unicode2.c \ 2161 2161 $(TOP)\ext\fts5\fts5_varint.c \ 2162 2162 $(TOP)\ext\fts5\fts5_vocab.c 2163 2163 2164 +LSM1_SRC = \ 2165 + $(TOP)\ext\lsm1\lsm.h \ 2166 + $(TOP)\ext\lsm1\lsmInt.h \ 2167 + $(TOP)\ext\lsm1\lsm_ckpt.c \ 2168 + $(TOP)\ext\lsm1\lsm_file.c \ 2169 + $(TOP)\ext\lsm1\lsm_log.c \ 2170 + $(TOP)\ext\lsm1\lsm_main.c \ 2171 + $(TOP)\ext\lsm1\lsm_mem.c \ 2172 + $(TOP)\ext\lsm1\lsm_mutex.c \ 2173 + $(TOP)\ext\lsm1\lsm_shared.c \ 2174 + $(TOP)\ext\lsm1\lsm_sorted.c \ 2175 + $(TOP)\ext\lsm1\lsm_str.c \ 2176 + $(TOP)\ext\lsm1\lsm_tree.c \ 2177 + $(TOP)\ext\lsm1\lsm_unix.c \ 2178 + $(TOP)\ext\lsm1\lsm_varint.c \ 2179 + $(TOP)\ext\lsm1\lsm_vtab.c \ 2180 + $(TOP)\ext\lsm1\lsm_win32.c 2181 + 2164 2182 fts5parse.c: $(TOP)\ext\fts5\fts5parse.y lemon.exe 2165 2183 copy $(TOP)\ext\fts5\fts5parse.y . 2166 2184 del /Q fts5parse.h 2>NUL 2167 2185 .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) fts5parse.y 2168 2186 2169 2187 fts5parse.h: fts5parse.c 2170 2188 2171 2189 fts5.c: $(FTS5_SRC) 2172 2190 $(TCLSH_CMD) $(TOP)\ext\fts5\tool\mkfts5c.tcl 2173 2191 copy $(TOP)\ext\fts5\fts5.h . 2174 2192 2193 +lsm1.c: $(LSM1_SRC) 2194 + $(TCLSH_CMD) $(TOP)\ext\lsm1\tool\mklsm1c.tcl 2195 + copy $(TOP)\ext\lsm1\lsm.h . 2196 + 2175 2197 fts5.lo: fts5.c $(HDR) $(EXTHDR) 2176 2198 $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c fts5.c 2177 2199 2178 2200 fts5_ext.lo: fts5.c $(HDR) $(EXTHDR) 2179 2201 $(LTCOMPILE) $(NO_WARN) -c fts5.c 2180 2202 2181 2203 fts5.dll: fts5_ext.lo ................................................................................ 2397 2419 del /Q sqlite3.c sqlite3-*.c 2>NUL 2398 2420 del /Q sqlite3rc.h 2>NUL 2399 2421 del /Q shell.c sqlite3ext.h sqlite3session.h 2>NUL 2400 2422 del /Q sqlite3_analyzer.exe sqlite3_analyzer.c 2>NUL 2401 2423 del /Q sqlite-*-output.vsix 2>NUL 2402 2424 del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe dbhash.exe 2>NUL 2403 2425 del /Q fts5.* fts5parse.* 2>NUL 2426 + del /Q lsm.h lsm1.c 2>NUL 2404 2427 # <</mark>>
Changes to README.md.
203 203 tool/mksqlite3c.tcl script is run to copy them all together in just the 204 204 right order while resolving internal "#include" references. 205 205 206 206 The amalgamation source file is more than 200K lines long. Some symbolic 207 207 debuggers (most notably MSVC) are unable to deal with files longer than 64K 208 208 lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl, 209 209 can be run on the amalgamation to break it up into a single small C file 210 -called **sqlite3-all.c** that does #include on about five other files 211 -named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-5.c**. In this way, 210 +called **sqlite3-all.c** that does #include on about seven other files 211 +named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-7.c**. In this way, 212 212 all of the source code is contained within a single translation unit so 213 213 that the compiler can do extra cross-procedure optimization, but no 214 214 individual source file exceeds 32K lines in length. 215 215 216 216 ## How It All Fits Together 217 217 218 218 SQLite is modular in design. ................................................................................ 233 233 Key files: 234 234 235 235 * **sqlite.h.in** - This file defines the public interface to the SQLite 236 236 library. Readers will need to be familiar with this interface before 237 237 trying to understand how the library works internally. 238 238 239 239 * **sqliteInt.h** - this header file defines many of the data objects 240 - used internally by SQLite. 240 + used internally by SQLite. In addition to "sqliteInt.h", some 241 + subsystems have their own header files. 241 242 242 243 * **parse.y** - This file describes the LALR(1) grammar that SQLite uses 243 244 to parse SQL statements, and the actions that are taken at each step 244 245 in the parsing process. 245 246 246 247 * **vdbe.c** - This file implements the virtual machine that runs 247 248 prepared statements. There are various helper files whose names 248 249 begin with "vdbe". The VDBE has access to the vdbeInt.h header file 249 250 which defines internal data objects. The rest of SQLite interacts 250 251 with the VDBE through an interface defined by vdbe.h. 251 252 252 - * **where.c** - This file analyzes the WHERE clause and generates 253 + * **where.c** - This file (together with its helper files named 254 + by "where*.c") analyzes the WHERE clause and generates 253 255 virtual machine code to run queries efficiently. This file is 254 256 sometimes called the "query optimizer". It has its own private 255 257 header file, whereInt.h, that defines data objects used internally. 256 258 257 259 * **btree.c** - This file contains the implementation of the B-Tree 258 - storage engine used by SQLite. 260 + storage engine used by SQLite. The interface to the rest of the system 261 + is defined by "btree.h". The "btreeInt.h" header defines objects 262 + used internally by btree.c and not published to the rest of the system. 259 263 260 264 * **pager.c** - This file contains the "pager" implementation, the 261 - module that implements transactions. 265 + module that implements transactions. The "pager.h" header file 266 + defines the interface between pager.c and the rest of the system. 262 267 263 268 * **os_unix.c** and **os_win.c** - These two files implement the interface 264 269 between SQLite and the underlying operating system using the run-time 265 270 pluggable VFS interface. 266 271 267 - * **shell.c** - This file is not part of the core SQLite library. This 272 + * **shell.c.in** - This file is not part of the core SQLite library. This 268 273 is the file that, when linked against sqlite3.a, generates the 269 - "sqlite3.exe" command-line shell. 274 + "sqlite3.exe" command-line shell. The "shell.c.in" file is transformed 275 + into "shell.c" as part of the build process. 270 276 271 277 * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It 272 278 is not part of the core SQLite library. But as most of the tests in this 273 279 repository are written in Tcl, the Tcl language bindings are important. 280 + 281 + * **test*.c** - Files in the src/ folder that begin with "test" go into 282 + building the "testfixture.exe" program. The testfixture.exe program is 283 + an enhanced TCL shell. The testfixture.exe program runs scripts in the 284 + test/ folder to validate the core SQLite code. The testfixture program 285 + (and some other test programs too) is build and run when you type 286 + "make test". 287 + 288 + * **ext/misc/json1.c** - This file implements the various JSON functions 289 + that are build into SQLite. 274 290 275 291 There are many other source files. Each has a succinct header comment that 276 292 describes its purpose and role within the larger system. 277 293 278 294 279 295 ## Contacts 280 296 281 297 The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/) 282 298 with geographically distributed backups at 283 299 [http://www2.sqlite.org/](http://www2.sqlite.org) and 284 300 [http://www3.sqlite.org/](http://www3.sqlite.org).
Changes to ext/fts5/fts5_index.c.
4905 4905 if( p2->n ){ 4906 4906 i64 iLastRowid = 0; 4907 4907 Fts5DoclistIter i1; 4908 4908 Fts5DoclistIter i2; 4909 4909 Fts5Buffer out = {0, 0, 0}; 4910 4910 Fts5Buffer tmp = {0, 0, 0}; 4911 4911 4912 - if( sqlite3Fts5BufferSize(&p->rc, &out, p1->n + p2->n) ) return; 4912 + /* The maximum size of the output is equal to the sum of the two 4913 + ** input sizes + 1 varint (9 bytes). The extra varint is because if the 4914 + ** first rowid in one input is a large negative number, and the first in 4915 + ** the other a non-negative number, the delta for the non-negative 4916 + ** number will be larger on disk than the literal integer value 4917 + ** was. */ 4918 + if( sqlite3Fts5BufferSize(&p->rc, &out, p1->n + p2->n + 9) ) return; 4913 4919 fts5DoclistIterInit(p1, &i1); 4914 4920 fts5DoclistIterInit(p2, &i2); 4915 4921 4916 4922 while( 1 ){ 4917 4923 if( i1.iRowid<i2.iRowid ){ 4918 4924 /* Copy entry from i1 */ 4919 4925 fts5MergeAppendDocid(&out, iLastRowid, i1.iRowid); ................................................................................ 4999 5005 fts5MergeAppendDocid(&out, iLastRowid, i1.iRowid); 5000 5006 fts5BufferSafeAppendBlob(&out, i1.aPoslist, i1.aEof - i1.aPoslist); 5001 5007 } 5002 5008 else if( i2.aPoslist ){ 5003 5009 fts5MergeAppendDocid(&out, iLastRowid, i2.iRowid); 5004 5010 fts5BufferSafeAppendBlob(&out, i2.aPoslist, i2.aEof - i2.aPoslist); 5005 5011 } 5012 + assert( out.n<=(p1->n+p2->n+9) ); 5006 5013 5007 5014 fts5BufferSet(&p->rc, p1, out.n, out.p); 5008 5015 fts5BufferFree(&tmp); 5009 5016 fts5BufferFree(&out); 5010 5017 } 5011 5018 } 5012 5019
Changes to ext/fts5/test/fts5query.test.
60 60 foreach x [list bbb ddd fff hhh jjj lll nnn ppp rrr ttt] { 61 61 set doc [string repeat "$x " 30] 62 62 execsql { INSERT INTO t1 VALUES($doc) } 63 63 } 64 64 execsql COMMIT 65 65 } {} 66 66 67 - do_execsql_test 1.$tn.2 { 67 + do_execsql_test 2.$tn.2 { 68 68 INSERT INTO t1(t1) VALUES('integrity-check'); 69 69 } 70 70 71 71 set ret 1 72 72 foreach x [list a c e g i k m o q s u] { 73 73 do_execsql_test 2.$tn.3.$ret { 74 74 SELECT rowid FROM t1 WHERE t1 MATCH $x || '*'; 75 75 } {} 76 76 incr ret 77 77 } 78 78 } 79 79 80 +reset_db 81 +do_execsql_test 3.0 { 82 + CREATE VIRTUAL TABLE x1 USING fts5(a); 83 + INSERT INTO x1(rowid, a) VALUES(-1000000000000, 'toyota'); 84 + INSERT INTO x1(rowid, a) VALUES(1, 'tarago'); 85 +} 86 +do_execsql_test 3.1 { 87 + SELECT rowid FROM x1('t*'); 88 +} {-1000000000000 1} 89 + 80 90 81 91 finish_test
Changes to ext/icu/icu.c.
24 24 ** 25 25 ** * Integration of ICU and SQLite collation sequences. 26 26 ** 27 27 ** * An implementation of the LIKE operator that uses ICU to 28 28 ** provide case-independent matching. 29 29 */ 30 30 31 -#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) 31 +#if !defined(SQLITE_CORE) \ 32 + || defined(SQLITE_ENABLE_ICU) \ 33 + || defined(SQLITE_ENABLE_ICU_COLLATIONS) 32 34 33 35 /* Include ICU headers */ 34 36 #include <unicode/utypes.h> 35 37 #include <unicode/uregex.h> 36 38 #include <unicode/ustring.h> 37 39 #include <unicode/ucol.h> 38 40 ................................................................................ 41 43 #ifndef SQLITE_CORE 42 44 #include "sqlite3ext.h" 43 45 SQLITE_EXTENSION_INIT1 44 46 #else 45 47 #include "sqlite3.h" 46 48 #endif 47 49 50 +/* 51 +** This function is called when an ICU function called from within 52 +** the implementation of an SQL scalar function returns an error. 53 +** 54 +** The scalar function context passed as the first argument is 55 +** loaded with an error message based on the following two args. 56 +*/ 57 +static void icuFunctionError( 58 + sqlite3_context *pCtx, /* SQLite scalar function context */ 59 + const char *zName, /* Name of ICU function that failed */ 60 + UErrorCode e /* Error code returned by ICU function */ 61 +){ 62 + char zBuf[128]; 63 + sqlite3_snprintf(128, zBuf, "ICU error: %s(): %s", zName, u_errorName(e)); 64 + zBuf[127] = '\0'; 65 + sqlite3_result_error(pCtx, zBuf, -1); 66 +} 67 + 68 +#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) 69 + 48 70 /* 49 71 ** Maximum length (in bytes) of the pattern in a LIKE or GLOB 50 72 ** operator. 51 73 */ 52 74 #ifndef SQLITE_MAX_LIKE_PATTERN_LENGTH 53 75 # define SQLITE_MAX_LIKE_PATTERN_LENGTH 50000 54 76 #endif ................................................................................ 220 242 } 221 243 222 244 if( zA && zB ){ 223 245 sqlite3_result_int(context, icuLikeCompare(zA, zB, uEsc)); 224 246 } 225 247 } 226 248 227 -/* 228 -** This function is called when an ICU function called from within 229 -** the implementation of an SQL scalar function returns an error. 230 -** 231 -** The scalar function context passed as the first argument is 232 -** loaded with an error message based on the following two args. 233 -*/ 234 -static void icuFunctionError( 235 - sqlite3_context *pCtx, /* SQLite scalar function context */ 236 - const char *zName, /* Name of ICU function that failed */ 237 - UErrorCode e /* Error code returned by ICU function */ 238 -){ 239 - char zBuf[128]; 240 - sqlite3_snprintf(128, zBuf, "ICU error: %s(): %s", zName, u_errorName(e)); 241 - zBuf[127] = '\0'; 242 - sqlite3_result_error(pCtx, zBuf, -1); 243 -} 244 - 245 249 /* 246 250 ** Function to delete compiled regexp objects. Registered as 247 251 ** a destructor function with sqlite3_set_auxdata(). 248 252 */ 249 253 static void icuRegexpDelete(void *p){ 250 254 URegularExpression *pExpr = (URegularExpression *)p; 251 255 uregex_close(pExpr); ................................................................................ 403 407 icuFunctionError(p, bToUpper ? "u_strToUpper" : "u_strToLower", status); 404 408 } 405 409 return; 406 410 } 407 411 assert( 0 ); /* Unreachable */ 408 412 } 409 413 414 +#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) */ 415 + 410 416 /* 411 417 ** Collation sequence destructor function. The pCtx argument points to 412 418 ** a UCollator structure previously allocated using ucol_open(). 413 419 */ 414 420 static void icuCollationDel(void *pCtx){ 415 421 UCollator *p = (UCollator *)pCtx; 416 422 ucol_close(p); ................................................................................ 497 503 const char *zName; /* Function name */ 498 504 unsigned char nArg; /* Number of arguments */ 499 505 unsigned short enc; /* Optimal text encoding */ 500 506 unsigned char iContext; /* sqlite3_user_data() context */ 501 507 void (*xFunc)(sqlite3_context*,int,sqlite3_value**); 502 508 } scalars[] = { 503 509 {"icu_load_collation", 2, SQLITE_UTF8, 1, icuLoadCollation}, 510 +#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) 504 511 {"regexp", 2, SQLITE_ANY|SQLITE_DETERMINISTIC, 0, icuRegexpFunc}, 505 512 {"lower", 1, SQLITE_UTF16|SQLITE_DETERMINISTIC, 0, icuCaseFunc16}, 506 513 {"lower", 2, SQLITE_UTF16|SQLITE_DETERMINISTIC, 0, icuCaseFunc16}, 507 514 {"upper", 1, SQLITE_UTF16|SQLITE_DETERMINISTIC, 1, icuCaseFunc16}, 508 515 {"upper", 2, SQLITE_UTF16|SQLITE_DETERMINISTIC, 1, icuCaseFunc16}, 509 516 {"lower", 1, SQLITE_UTF8|SQLITE_DETERMINISTIC, 0, icuCaseFunc16}, 510 517 {"lower", 2, SQLITE_UTF8|SQLITE_DETERMINISTIC, 0, icuCaseFunc16}, 511 518 {"upper", 1, SQLITE_UTF8|SQLITE_DETERMINISTIC, 1, icuCaseFunc16}, 512 519 {"upper", 2, SQLITE_UTF8|SQLITE_DETERMINISTIC, 1, icuCaseFunc16}, 513 520 {"like", 2, SQLITE_UTF8|SQLITE_DETERMINISTIC, 0, icuLikeFunc}, 514 521 {"like", 3, SQLITE_UTF8|SQLITE_DETERMINISTIC, 0, icuLikeFunc}, 522 +#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) */ 515 523 }; 516 524 int rc = SQLITE_OK; 517 525 int i; 518 - 519 526 520 527 for(i=0; rc==SQLITE_OK && i<(int)(sizeof(scalars)/sizeof(scalars[0])); i++){ 521 528 const struct IcuScalar *p = &scalars[i]; 522 529 rc = sqlite3_create_function( 523 530 db, p->zName, p->nArg, p->enc, 524 531 p->iContext ? (void*)db : (void*)0, 525 532 p->xFunc, 0, 0
Changes to ext/lsm1/lsmInt.h.
106 106 typedef unsigned short int u16; 107 107 typedef unsigned int u32; 108 108 typedef lsm_i64 i64; 109 109 typedef unsigned long long int u64; 110 110 #endif 111 111 112 112 /* A page number is a 64-bit integer. */ 113 -typedef i64 Pgno; 113 +typedef i64 LsmPgno; 114 114 115 115 #ifdef LSM_DEBUG 116 116 int lsmErrorBkpt(int); 117 117 #else 118 118 # define lsmErrorBkpt(x) (x) 119 119 #endif 120 120 ................................................................................ 398 398 void **apShm; /* Shared memory chunks */ 399 399 ShmHeader *pShmhdr; /* Live shared-memory header */ 400 400 TreeHeader treehdr; /* Local copy of tree-header */ 401 401 u32 aSnapshot[LSM_META_PAGE_SIZE / sizeof(u32)]; 402 402 }; 403 403 404 404 struct Segment { 405 - Pgno iFirst; /* First page of this run */ 406 - Pgno iLastPg; /* Last page of this run */ 407 - Pgno iRoot; /* Root page number (if any) */ 405 + LsmPgno iFirst; /* First page of this run */ 406 + LsmPgno iLastPg; /* Last page of this run */ 407 + LsmPgno iRoot; /* Root page number (if any) */ 408 408 int nSize; /* Size of this run in pages */ 409 409 410 410 Redirect *pRedirect; /* Block redirects (or NULL) */ 411 411 }; 412 412 413 413 /* 414 414 ** iSplitTopic/pSplitKey/nSplitKey: ................................................................................ 452 452 ** access to the associated Level struct. 453 453 ** 454 454 ** iOutputOff: 455 455 ** The byte offset to write to next within the last page of the 456 456 ** output segment. 457 457 */ 458 458 struct MergeInput { 459 - Pgno iPg; /* Page on which next input is stored */ 459 + LsmPgno iPg; /* Page on which next input is stored */ 460 460 int iCell; /* Cell containing next input to merge */ 461 461 }; 462 462 struct Merge { 463 463 int nInput; /* Number of input runs being merged */ 464 464 MergeInput *aInput; /* Array nInput entries in size */ 465 465 MergeInput splitkey; /* Location in file of current splitkey */ 466 466 int nSkip; /* Number of separators entries to skip */ 467 467 int iOutputOff; /* Write offset on output page */ 468 - Pgno iCurrentPtr; /* Current pointer value */ 468 + LsmPgno iCurrentPtr; /* Current pointer value */ 469 469 }; 470 470 471 471 /* 472 472 ** The first argument to this macro is a pointer to a Segment structure. 473 473 ** Returns true if the structure instance indicates that the separators 474 474 ** array is valid. 475 475 */ ................................................................................ 575 575 u32 iCmpId; /* Id of compression scheme */ 576 576 Level *pLevel; /* Pointer to level 0 of snapshot (or NULL) */ 577 577 i64 iId; /* Snapshot id */ 578 578 i64 iLogOff; /* Log file offset */ 579 579 Redirect redirect; /* Block redirection array */ 580 580 581 581 /* Used by worker snapshots only */ 582 - int nBlock; /* Number of blocks in database file */ 583 - Pgno aiAppend[LSM_APPLIST_SZ]; /* Append point list */ 584 - Freelist freelist; /* Free block list */ 585 - u32 nWrite; /* Total number of pages written to disk */ 582 + int nBlock; /* Number of blocks in database file */ 583 + LsmPgno aiAppend[LSM_APPLIST_SZ]; /* Append point list */ 584 + Freelist freelist; /* Free block list */ 585 + u32 nWrite; /* Total number of pages written to disk */ 586 586 }; 587 587 #define LSM_INITIAL_SNAPSHOT_ID 11 588 588 589 589 /* 590 590 ** Functions from file "lsm_ckpt.c". 591 591 */ 592 592 int lsmCheckpointWrite(lsm_db *, u32 *); ................................................................................ 706 706 707 707 int lsmFsPageSize(FileSystem *); 708 708 void lsmFsSetPageSize(FileSystem *, int); 709 709 710 710 int lsmFsFileid(lsm_db *pDb, void **ppId, int *pnId); 711 711 712 712 /* Creating, populating, gobbling and deleting sorted runs. */ 713 -void lsmFsGobble(lsm_db *, Segment *, Pgno *, int); 713 +void lsmFsGobble(lsm_db *, Segment *, LsmPgno *, int); 714 714 int lsmFsSortedDelete(FileSystem *, Snapshot *, int, Segment *); 715 715 int lsmFsSortedFinish(FileSystem *, Segment *); 716 716 int lsmFsSortedAppend(FileSystem *, Snapshot *, Level *, int, Page **); 717 717 int lsmFsSortedPadding(FileSystem *, Snapshot *, Segment *); 718 718 719 719 /* Functions to retrieve the lsm_env pointer from a FileSystem or Page object */ 720 720 lsm_env *lsmFsEnv(FileSystem *); ................................................................................ 723 723 724 724 int lsmFsSectorSize(FileSystem *); 725 725 726 726 void lsmSortedSplitkey(lsm_db *, Level *, int *); 727 727 728 728 /* Reading sorted run content. */ 729 729 int lsmFsDbPageLast(FileSystem *pFS, Segment *pSeg, Page **ppPg); 730 -int lsmFsDbPageGet(FileSystem *, Segment *, Pgno, Page **); 730 +int lsmFsDbPageGet(FileSystem *, Segment *, LsmPgno, Page **); 731 731 int lsmFsDbPageNext(Segment *, Page *, int eDir, Page **); 732 732 733 733 u8 *lsmFsPageData(Page *, int *); 734 734 int lsmFsPageRelease(Page *); 735 735 int lsmFsPagePersist(Page *); 736 736 void lsmFsPageRef(Page *); 737 -Pgno lsmFsPageNumber(Page *); 737 +LsmPgno lsmFsPageNumber(Page *); 738 738 739 739 int lsmFsNRead(FileSystem *); 740 740 int lsmFsNWrite(FileSystem *); 741 741 742 742 int lsmFsMetaPageGet(FileSystem *, int, int, MetaPage **); 743 743 int lsmFsMetaPageRelease(MetaPage *); 744 744 u8 *lsmFsMetaPageData(MetaPage *, int *); 745 745 746 746 #ifdef LSM_DEBUG 747 747 int lsmFsDbPageIsLast(Segment *pSeg, Page *pPg); 748 748 int lsmFsIntegrityCheck(lsm_db *); 749 749 #endif 750 750 751 -Pgno lsmFsRedirectPage(FileSystem *, Redirect *, Pgno); 751 +LsmPgno lsmFsRedirectPage(FileSystem *, Redirect *, LsmPgno); 752 752 753 753 int lsmFsPageWritable(Page *); 754 754 755 755 /* Functions to read, write and sync the log file. */ 756 756 int lsmFsWriteLog(FileSystem *pFS, i64 iOff, LsmString *pStr); 757 757 int lsmFsSyncLog(FileSystem *pFS); 758 758 int lsmFsReadLog(FileSystem *pFS, i64 iOff, int nRead, LsmString *pStr); ................................................................................ 764 764 765 765 /* And to sync the db file */ 766 766 int lsmFsSyncDb(FileSystem *, int); 767 767 768 768 void lsmFsFlushWaiting(FileSystem *, int *); 769 769 770 770 /* Used by lsm_info(ARRAY_STRUCTURE) and lsm_config(MMAP) */ 771 -int lsmInfoArrayStructure(lsm_db *pDb, int bBlock, Pgno iFirst, char **pzOut); 772 -int lsmInfoArrayPages(lsm_db *pDb, Pgno iFirst, char **pzOut); 771 +int lsmInfoArrayStructure(lsm_db *pDb, int bBlock, LsmPgno iFirst, char **pz); 772 +int lsmInfoArrayPages(lsm_db *pDb, LsmPgno iFirst, char **pzOut); 773 773 int lsmConfigMmap(lsm_db *pDb, int *piParam); 774 774 775 775 int lsmEnvOpen(lsm_env *, const char *, int, lsm_file **); 776 776 int lsmEnvClose(lsm_env *pEnv, lsm_file *pFile); 777 777 int lsmEnvLock(lsm_env *pEnv, lsm_file *pFile, int iLock, int eLock); 778 778 int lsmEnvTestLock(lsm_env *pEnv, lsm_file *pFile, int iLock, int nLock, int); 779 779 ................................................................................ 781 781 void lsmEnvShmBarrier(lsm_env *); 782 782 void lsmEnvShmUnmap(lsm_env *, lsm_file *, int); 783 783 784 784 void lsmEnvSleep(lsm_env *, int); 785 785 786 786 int lsmFsReadSyncedId(lsm_db *db, int, i64 *piVal); 787 787 788 -int lsmFsSegmentContainsPg(FileSystem *pFS, Segment *, Pgno, int *); 788 +int lsmFsSegmentContainsPg(FileSystem *pFS, Segment *, LsmPgno, int *); 789 789 790 790 void lsmFsPurgeCache(FileSystem *); 791 791 792 792 /* 793 793 ** End of functions from "lsm_file.c". 794 794 **************************************************************************/ 795 795 796 796 /* 797 797 ** Functions from file "lsm_sorted.c". 798 798 */ 799 -int lsmInfoPageDump(lsm_db *, Pgno, int, char **); 799 +int lsmInfoPageDump(lsm_db *, LsmPgno, int, char **); 800 800 void lsmSortedCleanup(lsm_db *); 801 801 int lsmSortedAutoWork(lsm_db *, int nUnit); 802 802 803 803 int lsmSortedWalkFreelist(lsm_db *, int, int (*)(void *, int, i64), void *); 804 804 805 805 int lsmSaveWorker(lsm_db *, int); 806 806
Changes to ext/lsm1/lsm_ckpt.c.
385 385 static void ckptExportAppendlist( 386 386 lsm_db *db, /* Database connection */ 387 387 CkptBuffer *p, /* Checkpoint buffer to write to */ 388 388 int *piOut, /* IN/OUT: Offset within checkpoint buffer */ 389 389 int *pRc /* IN/OUT: Error code */ 390 390 ){ 391 391 int i; 392 - Pgno *aiAppend = db->pWorker->aiAppend; 392 + LsmPgno *aiAppend = db->pWorker->aiAppend; 393 393 394 394 for(i=0; i<LSM_APPLIST_SZ; i++){ 395 395 ckptAppend64(p, piOut, aiAppend[i], pRc); 396 396 } 397 397 }; 398 398 399 399 static int ckptExportSnapshot(
Changes to ext/lsm1/lsm_file.c.
265 265 ** The lsmFsSortedAppend() function sets the pSeg pointer to point to the 266 266 ** segment that the new page will be a part of. It is unset by 267 267 ** lsmFsPagePersist() after the page is written to disk. 268 268 */ 269 269 struct Page { 270 270 u8 *aData; /* Buffer containing page data */ 271 271 int nData; /* Bytes of usable data at aData[] */ 272 - Pgno iPg; /* Page number */ 272 + LsmPgno iPg; /* Page number */ 273 273 int nRef; /* Number of outstanding references */ 274 274 int flags; /* Combination of PAGE_XXX flags */ 275 275 Page *pHashNext; /* Next page in hash table slot */ 276 276 Page *pLruNext; /* Next page in LRU list */ 277 277 Page *pLruPrev; /* Previous page in LRU list */ 278 278 FileSystem *pFS; /* File system that owns this page */ 279 279 ................................................................................ 328 328 #else 329 329 # define IOERR_WRAPPER(rc) (rc) 330 330 #endif 331 331 332 332 #ifdef NDEBUG 333 333 # define assert_lists_are_ok(x) 334 334 #else 335 -static Page *fsPageFindInHash(FileSystem *pFS, Pgno iPg, int *piHash); 335 +static Page *fsPageFindInHash(FileSystem *pFS, LsmPgno iPg, int *piHash); 336 336 337 337 static void assert_lists_are_ok(FileSystem *pFS){ 338 338 #if 0 339 339 Page *p; 340 340 341 341 assert( pFS->nMapLimit>=0 ); 342 342 ................................................................................ 528 528 return LSM_OK; 529 529 } 530 530 531 531 /* 532 532 ** Return true if page iReal of the database should be accessed using mmap. 533 533 ** False otherwise. 534 534 */ 535 -static int fsMmapPage(FileSystem *pFS, Pgno iReal){ 535 +static int fsMmapPage(FileSystem *pFS, LsmPgno iReal){ 536 536 return ((i64)iReal*pFS->nPagesize <= pFS->nMapLimit); 537 537 } 538 538 539 539 /* 540 540 ** Given that there are currently nHash slots in the hash table, return 541 541 ** the hash key for file iFile, page iPg. 542 542 */ 543 -static int fsHashKey(int nHash, Pgno iPg){ 543 +static int fsHashKey(int nHash, LsmPgno iPg){ 544 544 return (iPg % nHash); 545 545 } 546 546 547 547 /* 548 548 ** This is a helper function for lsmFsOpen(). It opens a single file on 549 549 ** disk (either the database or log file). 550 550 */ ................................................................................ 876 876 ** Return the page number of the first page on block iBlock. Blocks are 877 877 ** numbered starting from 1. 878 878 ** 879 879 ** For a compressed database, page numbers are byte offsets. The first 880 880 ** page on each block is the byte offset immediately following the 4-byte 881 881 ** "previous block" pointer at the start of each block. 882 882 */ 883 -static Pgno fsFirstPageOnBlock(FileSystem *pFS, int iBlock){ 884 - Pgno iPg; 883 +static LsmPgno fsFirstPageOnBlock(FileSystem *pFS, int iBlock){ 884 + LsmPgno iPg; 885 885 if( pFS->pCompress ){ 886 886 if( iBlock==1 ){ 887 887 iPg = pFS->nMetasize * 2 + 4; 888 888 }else{ 889 - iPg = pFS->nBlocksize * (Pgno)(iBlock-1) + 4; 889 + iPg = pFS->nBlocksize * (LsmPgno)(iBlock-1) + 4; 890 890 } 891 891 }else{ 892 892 const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize); 893 893 if( iBlock==1 ){ 894 894 iPg = 1 + ((pFS->nMetasize*2 + pFS->nPagesize - 1) / pFS->nPagesize); 895 895 }else{ 896 896 iPg = 1 + (iBlock-1) * nPagePerBlock; ................................................................................ 903 903 ** Return the page number of the last page on block iBlock. Blocks are 904 904 ** numbered starting from 1. 905 905 ** 906 906 ** For a compressed database, page numbers are byte offsets. The first 907 907 ** page on each block is the byte offset of the byte immediately before 908 908 ** the 4-byte "next block" pointer at the end of each block. 909 909 */ 910 -static Pgno fsLastPageOnBlock(FileSystem *pFS, int iBlock){ 910 +static LsmPgno fsLastPageOnBlock(FileSystem *pFS, int iBlock){ 911 911 if( pFS->pCompress ){ 912 - return pFS->nBlocksize * (Pgno)iBlock - 1 - 4; 912 + return pFS->nBlocksize * (LsmPgno)iBlock - 1 - 4; 913 913 }else{ 914 914 const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize); 915 915 return iBlock * nPagePerBlock; 916 916 } 917 917 } 918 918 919 919 /* 920 920 ** Return the block number of the block that page iPg is located on. 921 921 ** Blocks are numbered starting from 1. 922 922 */ 923 -static int fsPageToBlock(FileSystem *pFS, Pgno iPg){ 923 +static int fsPageToBlock(FileSystem *pFS, LsmPgno iPg){ 924 924 if( pFS->pCompress ){ 925 925 return (int)((iPg / pFS->nBlocksize) + 1); 926 926 }else{ 927 927 return (int)(1 + ((iPg-1) / (pFS->nBlocksize / pFS->nPagesize))); 928 928 } 929 929 } 930 930 931 931 /* 932 932 ** Return true if page iPg is the last page on its block. 933 933 ** 934 934 ** This function is only called in non-compressed database mode. 935 935 */ 936 -static int fsIsLast(FileSystem *pFS, Pgno iPg){ 936 +static int fsIsLast(FileSystem *pFS, LsmPgno iPg){ 937 937 const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize); 938 938 assert( !pFS->pCompress ); 939 939 return ( iPg && (iPg % nPagePerBlock)==0 ); 940 940 } 941 941 942 942 /* 943 943 ** Return true if page iPg is the first page on its block. 944 944 ** 945 945 ** This function is only called in non-compressed database mode. 946 946 */ 947 -static int fsIsFirst(FileSystem *pFS, Pgno iPg){ 947 +static int fsIsFirst(FileSystem *pFS, LsmPgno iPg){ 948 948 const int nPagePerBlock = (pFS->nBlocksize / pFS->nPagesize); 949 949 assert( !pFS->pCompress ); 950 950 return ( (iPg % nPagePerBlock)==1 951 951 || (iPg<nPagePerBlock && iPg==fsFirstPageOnBlock(pFS, 1)) 952 952 ); 953 953 } 954 954 ................................................................................ 963 963 } 964 964 return pPage->aData; 965 965 } 966 966 967 967 /* 968 968 ** Return the page number of a page. 969 969 */ 970 -Pgno lsmFsPageNumber(Page *pPage){ 970 +LsmPgno lsmFsPageNumber(Page *pPage){ 971 971 /* assert( (pPage->flags & PAGE_DIRTY)==0 ); */ 972 972 return pPage ? pPage->iPg : 0; 973 973 } 974 974 975 975 /* 976 976 ** Page pPg is currently part of the LRU list belonging to pFS. Remove 977 977 ** it from the list. pPg->pLruNext and pPg->pLruPrev are cleared by this ................................................................................ 1054 1054 /* 1055 1055 ** Search the hash-table for page iPg. If an entry is round, return a pointer 1056 1056 ** to it. Otherwise, return NULL. 1057 1057 ** 1058 1058 ** Either way, if argument piHash is not NULL set *piHash to the hash slot 1059 1059 ** number that page iPg would be stored in before returning. 1060 1060 */ 1061 -static Page *fsPageFindInHash(FileSystem *pFS, Pgno iPg, int *piHash){ 1061 +static Page *fsPageFindInHash(FileSystem *pFS, LsmPgno iPg, int *piHash){ 1062 1062 Page *p; /* Return value */ 1063 1063 int iHash = fsHashKey(pFS->nHash, iPg); 1064 1064 1065 1065 if( piHash ) *piHash = iHash; 1066 1066 for(p=pFS->apHash[iHash]; p; p=p->pHashNext){ 1067 1067 if( p->iPg==iPg) break; 1068 1068 } ................................................................................ 1185 1185 } 1186 1186 1187 1187 /* 1188 1188 ** If page iPg has been redirected according to the redirections in the 1189 1189 ** object passed as the second argument, return the destination page to 1190 1190 ** which it is redirected. Otherwise, return a copy of iPg. 1191 1191 */ 1192 -Pgno lsmFsRedirectPage(FileSystem *pFS, Redirect *pRedir, Pgno iPg){ 1193 - Pgno iReal = iPg; 1192 +LsmPgno lsmFsRedirectPage(FileSystem *pFS, Redirect *pRedir, LsmPgno iPg){ 1193 + LsmPgno iReal = iPg; 1194 1194 1195 1195 if( pRedir ){ 1196 1196 const int nPagePerBlock = ( 1197 1197 pFS->pCompress ? pFS->nBlocksize : (pFS->nBlocksize / pFS->nPagesize) 1198 1198 ); 1199 1199 int iBlk = fsPageToBlock(pFS, iPg); 1200 1200 int i; 1201 1201 for(i=0; i<pRedir->n; i++){ 1202 1202 int iFrom = pRedir->a[i].iFrom; 1203 1203 if( iFrom>iBlk ) break; 1204 1204 if( iFrom==iBlk ){ 1205 1205 int iTo = pRedir->a[i].iTo; 1206 - iReal = iPg - (Pgno)(iFrom - iTo) * nPagePerBlock; 1206 + iReal = iPg - (LsmPgno)(iFrom - iTo) * nPagePerBlock; 1207 1207 if( iTo==1 ){ 1208 1208 iReal += (fsFirstPageOnBlock(pFS, 1)-1); 1209 1209 } 1210 1210 break; 1211 1211 } 1212 1212 } 1213 1213 } 1214 1214 1215 1215 assert( iReal!=0 ); 1216 1216 return iReal; 1217 1217 } 1218 1218 1219 1219 /* Required by the circular fsBlockNext<->fsPageGet dependency. */ 1220 -static int fsPageGet(FileSystem *, Segment *, Pgno, int, Page **, int *); 1220 +static int fsPageGet(FileSystem *, Segment *, LsmPgno, int, Page **, int *); 1221 1221 1222 1222 /* 1223 1223 ** Parameter iBlock is a database file block. This function reads the value 1224 1224 ** stored in the blocks "next block" pointer and stores it in *piNext. 1225 1225 ** LSM_OK is returned if everything is successful, or an LSM error code 1226 1226 ** otherwise. 1227 1227 */ ................................................................................ 1265 1265 } 1266 1266 return rc; 1267 1267 } 1268 1268 1269 1269 /* 1270 1270 ** Return the page number of the last page on the same block as page iPg. 1271 1271 */ 1272 -Pgno fsLastPageOnPagesBlock(FileSystem *pFS, Pgno iPg){ 1272 +LsmPgno fsLastPageOnPagesBlock(FileSystem *pFS, LsmPgno iPg){ 1273 1273 return fsLastPageOnBlock(pFS, fsPageToBlock(pFS, iPg)); 1274 1274 } 1275 1275 1276 1276 /* 1277 1277 ** Read nData bytes of data from offset iOff of the database file into 1278 1278 ** buffer aData. If this means reading past the end of a block, follow 1279 1279 ** the block pointer to the next block and continue reading. ................................................................................ 1533 1533 ** to the total number of free bytes before returning. 1534 1534 ** 1535 1535 ** If no error occurs, LSM_OK is returned. Otherwise, an lsm error code. 1536 1536 */ 1537 1537 static int fsPageGet( 1538 1538 FileSystem *pFS, /* File-system handle */ 1539 1539 Segment *pSeg, /* Block redirection to use (or NULL) */ 1540 - Pgno iPg, /* Page id */ 1540 + LsmPgno iPg, /* Page id */ 1541 1541 int noContent, /* True to not load content from disk */ 1542 1542 Page **ppPg, /* OUT: New page handle */ 1543 1543 int *pnSpace /* OUT: Bytes of free space */ 1544 1544 ){ 1545 1545 Page *p; 1546 1546 int iHash; 1547 1547 int rc = LSM_OK; 1548 1548 1549 1549 /* In most cases iReal is the same as iPg. Except, if pSeg->pRedirect is 1550 1550 ** not NULL, and the block containing iPg has been redirected, then iReal 1551 1551 ** is the page number after redirection. */ 1552 - Pgno iReal = lsmFsRedirectPage(pFS, (pSeg ? pSeg->pRedirect : 0), iPg); 1552 + LsmPgno iReal = lsmFsRedirectPage(pFS, (pSeg ? pSeg->pRedirect : 0), iPg); 1553 1553 1554 1554 assert_lists_are_ok(pFS); 1555 1555 assert( iPg>=fsFirstPageOnBlock(pFS, 1) ); 1556 1556 assert( iReal>=fsFirstPageOnBlock(pFS, 1) ); 1557 1557 *ppPg = 0; 1558 1558 1559 1559 /* Search the hash-table for the page */ ................................................................................ 1685 1685 /* 1686 1686 ** Return true if the first or last page of segment pRun falls between iFirst 1687 1687 ** and iLast, inclusive, and pRun is not equal to pIgnore. 1688 1688 */ 1689 1689 static int fsRunEndsBetween( 1690 1690 Segment *pRun, 1691 1691 Segment *pIgnore, 1692 - Pgno iFirst, 1693 - Pgno iLast 1692 + LsmPgno iFirst, 1693 + LsmPgno iLast 1694 1694 ){ 1695 1695 return (pRun!=pIgnore && ( 1696 1696 (pRun->iFirst>=iFirst && pRun->iFirst<=iLast) 1697 1697 || (pRun->iLastPg>=iFirst && pRun->iLastPg<=iLast) 1698 1698 )); 1699 1699 } 1700 1700 ................................................................................ 1701 1701 /* 1702 1702 ** Return true if level pLevel contains a segment other than pIgnore for 1703 1703 ** which the first or last page is between iFirst and iLast, inclusive. 1704 1704 */ 1705 1705 static int fsLevelEndsBetween( 1706 1706 Level *pLevel, 1707 1707 Segment *pIgnore, 1708 - Pgno iFirst, 1709 - Pgno iLast 1708 + LsmPgno iFirst, 1709 + LsmPgno iLast 1710 1710 ){ 1711 1711 int i; 1712 1712 1713 1713 if( fsRunEndsBetween(&pLevel->lhs, pIgnore, iFirst, iLast) ){ 1714 1714 return 1; 1715 1715 } 1716 1716 for(i=0; i<pLevel->nRight; i++){ ................................................................................ 1729 1729 static int fsFreeBlock( 1730 1730 FileSystem *pFS, /* File system object */ 1731 1731 Snapshot *pSnapshot, /* Worker snapshot */ 1732 1732 Segment *pIgnore, /* Ignore this run when searching */ 1733 1733 int iBlk /* Block number of block to free */ 1734 1734 ){ 1735 1735 int rc = LSM_OK; /* Return code */ 1736 - Pgno iFirst; /* First page on block iBlk */ 1737 - Pgno iLast; /* Last page on block iBlk */ 1736 + LsmPgno iFirst; /* First page on block iBlk */ 1737 + LsmPgno iLast; /* Last page on block iBlk */ 1738 1738 Level *pLevel; /* Used to iterate through levels */ 1739 1739 1740 1740 int iIn; /* Used to iterate through append points */ 1741 1741 int iOut = 0; /* Used to output append points */ 1742 - Pgno *aApp = pSnapshot->aiAppend; 1742 + LsmPgno *aApp = pSnapshot->aiAppend; 1743 1743 1744 1744 iFirst = fsFirstPageOnBlock(pFS, iBlk); 1745 1745 iLast = fsLastPageOnBlock(pFS, iBlk); 1746 1746 1747 1747 /* Check if any other run in the snapshot has a start or end page 1748 1748 ** within this block. If there is such a run, return early. */ 1749 1749 for(pLevel=lsmDbSnapshotLevel(pSnapshot); pLevel; pLevel=pLevel->pNext){ ................................................................................ 1807 1807 } 1808 1808 1809 1809 /* 1810 1810 ** aPgno is an array containing nPgno page numbers. Return the smallest page 1811 1811 ** number from the array that falls on block iBlk. Or, if none of the pages 1812 1812 ** in aPgno[] fall on block iBlk, return 0. 1813 1813 */ 1814 -static Pgno firstOnBlock(FileSystem *pFS, int iBlk, Pgno *aPgno, int nPgno){ 1815 - Pgno iRet = 0; 1814 +static LsmPgno firstOnBlock( 1815 + FileSystem *pFS, 1816 + int iBlk, 1817 + LsmPgno *aPgno, 1818 + int nPgno 1819 +){ 1820 + LsmPgno iRet = 0; 1816 1821 int i; 1817 1822 for(i=0; i<nPgno; i++){ 1818 - Pgno iPg = aPgno[i]; 1823 + LsmPgno iPg = aPgno[i]; 1819 1824 if( fsPageToBlock(pFS, iPg)==iBlk && (iRet==0 || iPg<iRet) ){ 1820 1825 iRet = iPg; 1821 1826 } 1822 1827 } 1823 1828 return iRet; 1824 1829 } 1825 1830 1826 1831 #ifndef NDEBUG 1827 1832 /* 1828 1833 ** Return true if page iPg, which is a part of segment p, lies on 1829 1834 ** a redirected block. 1830 1835 */ 1831 -static int fsPageRedirects(FileSystem *pFS, Segment *p, Pgno iPg){ 1836 +static int fsPageRedirects(FileSystem *pFS, Segment *p, LsmPgno iPg){ 1832 1837 return (iPg!=0 && iPg!=lsmFsRedirectPage(pFS, p->pRedirect, iPg)); 1833 1838 } 1834 1839 1835 1840 /* 1836 1841 ** Return true if the second argument is not NULL and any of the first 1837 1842 ** last or root pages lie on a redirected block. 1838 1843 */ ................................................................................ 1850 1855 ** the segment pRun. This function gobbles from the start of the run to the 1851 1856 ** first page that appears in aPgno[] (i.e. so that the aPgno[] entry is 1852 1857 ** the new first page of the run). 1853 1858 */ 1854 1859 void lsmFsGobble( 1855 1860 lsm_db *pDb, 1856 1861 Segment *pRun, 1857 - Pgno *aPgno, 1862 + LsmPgno *aPgno, 1858 1863 int nPgno 1859 1864 ){ 1860 1865 int rc = LSM_OK; 1861 1866 FileSystem *pFS = pDb->pFS; 1862 1867 Snapshot *pSnapshot = pDb->pWorker; 1863 1868 int iBlk; 1864 1869 ................................................................................ 1867 1872 assert( nPgno>0 && 0==fsPageRedirects(pFS, pRun, aPgno[0]) ); 1868 1873 1869 1874 iBlk = fsPageToBlock(pFS, pRun->iFirst); 1870 1875 pRun->nSize += (int)(pRun->iFirst - fsFirstPageOnBlock(pFS, iBlk)); 1871 1876 1872 1877 while( rc==LSM_OK ){ 1873 1878 int iNext = 0; 1874 - Pgno iFirst = firstOnBlock(pFS, iBlk, aPgno, nPgno); 1879 + LsmPgno iFirst = firstOnBlock(pFS, iBlk, aPgno, nPgno); 1875 1880 if( iFirst ){ 1876 1881 pRun->iFirst = iFirst; 1877 1882 break; 1878 1883 } 1879 1884 rc = fsBlockNext(pFS, pRun, iBlk, &iNext); 1880 1885 if( rc==LSM_OK ) rc = fsFreeBlock(pFS, pSnapshot, pRun, iBlk); 1881 1886 pRun->nSize -= (int)( ................................................................................ 1901 1906 ** *piNext = iPg + nByte; 1902 1907 ** 1903 1908 ** But take block overflow and redirection into account. 1904 1909 */ 1905 1910 static int fsNextPageOffset( 1906 1911 FileSystem *pFS, /* File system object */ 1907 1912 Segment *pSeg, /* Segment to move within */ 1908 - Pgno iPg, /* Offset of current page */ 1913 + LsmPgno iPg, /* Offset of current page */ 1909 1914 int nByte, /* Size of current page including headers */ 1910 - Pgno *piNext /* OUT: Offset of next page. Or zero (EOF) */ 1915 + LsmPgno *piNext /* OUT: Offset of next page. Or zero (EOF) */ 1911 1916 ){ 1912 - Pgno iNext; 1917 + LsmPgno iNext; 1913 1918 int rc; 1914 1919 1915 1920 assert( pFS->pCompress ); 1916 1921 1917 1922 rc = fsAddOffset(pFS, pSeg, iPg, nByte-1, &iNext); 1918 1923 if( pSeg && iNext==pSeg->iLastPg ){ 1919 1924 iNext = 0; ................................................................................ 1935 1940 ** LSM_OK is returned if no error occurs. Otherwise, an lsm error code. 1936 1941 ** If any value other than LSM_OK is returned, then the final value of 1937 1942 ** *piPrev is undefined. 1938 1943 */ 1939 1944 static int fsGetPageBefore( 1940 1945 FileSystem *pFS, 1941 1946 Segment *pSeg, 1942 - Pgno iPg, 1943 - Pgno *piPrev 1947 + LsmPgno iPg, 1948 + LsmPgno *piPrev 1944 1949 ){ 1945 1950 u8 aSz[3]; 1946 1951 int rc; 1947 1952 i64 iRead; 1948 1953 1949 1954 assert( pFS->pCompress ); 1950 1955 ................................................................................ 1986 1991 ** 1987 1992 ** Page references returned by this function should be released by the 1988 1993 ** caller using lsmFsPageRelease(). 1989 1994 */ 1990 1995 int lsmFsDbPageNext(Segment *pRun, Page *pPg, int eDir, Page **ppNext){ 1991 1996 int rc = LSM_OK; 1992 1997 FileSystem *pFS = pPg->pFS; 1993 - Pgno iPg = pPg->iPg; 1998 + LsmPgno iPg = pPg->iPg; 1994 1999 1995 2000 assert( 0==fsSegmentRedirects(pFS, pRun) ); 1996 2001 if( pFS->pCompress ){ 1997 2002 int nSpace = pPg->nCompress + 2*3; 1998 2003 1999 2004 do { 2000 2005 if( eDir>0 ){ ................................................................................ 2058 2063 ** already allocated block. If it is possible, the page number of the first 2059 2064 ** page to use for the new segment is returned. Otherwise zero. 2060 2065 ** 2061 2066 ** If argument pLvl is not NULL, then this function will not attempt to 2062 2067 ** start the new segment immediately following any segment that is part 2063 2068 ** of the right-hand-side of pLvl. 2064 2069 */ 2065 -static Pgno findAppendPoint(FileSystem *pFS, Level *pLvl){ 2070 +static LsmPgno findAppendPoint(FileSystem *pFS, Level *pLvl){ 2066 2071 int i; 2067 - Pgno *aiAppend = pFS->pDb->pWorker->aiAppend; 2068 - Pgno iRet = 0; 2072 + LsmPgno *aiAppend = pFS->pDb->pWorker->aiAppend; 2073 + LsmPgno iRet = 0; 2069 2074 2070 2075 for(i=LSM_APPLIST_SZ-1; iRet==0 && i>=0; i--){ 2071 2076 if( (iRet = aiAppend[i]) ){ 2072 2077 if( pLvl ){ 2073 2078 int iBlk = fsPageToBlock(pFS, iRet); 2074 2079 int j; 2075 2080 for(j=0; iRet && j<pLvl->nRight; j++){ ................................................................................ 2094 2099 Snapshot *pSnapshot, 2095 2100 Level *pLvl, 2096 2101 int bDefer, 2097 2102 Page **ppOut 2098 2103 ){ 2099 2104 int rc = LSM_OK; 2100 2105 Page *pPg = 0; 2101 - Pgno iApp = 0; 2102 - Pgno iNext = 0; 2106 + LsmPgno iApp = 0; 2107 + LsmPgno iNext = 0; 2103 2108 Segment *p = &pLvl->lhs; 2104 - Pgno iPrev = p->iLastPg; 2109 + LsmPgno iPrev = p->iLastPg; 2105 2110 2106 2111 *ppOut = 0; 2107 2112 assert( p->pRedirect==0 ); 2108 2113 2109 2114 if( pFS->pCompress || bDefer ){ 2110 2115 /* In compressed database mode the page is not assigned a page number 2111 2116 ** or location in the database file at this point. This will be done ................................................................................ 2191 2196 ** Shift this extra block back to the free-block list. 2192 2197 ** 2193 2198 ** Otherwise, add the first free page in the last block used by the run 2194 2199 ** to the lAppend list. 2195 2200 */ 2196 2201 if( fsLastPageOnPagesBlock(pFS, p->iLastPg)!=p->iLastPg ){ 2197 2202 int i; 2198 - Pgno *aiAppend = pFS->pDb->pWorker->aiAppend; 2203 + LsmPgno *aiAppend = pFS->pDb->pWorker->aiAppend; 2199 2204 for(i=0; i<LSM_APPLIST_SZ; i++){ 2200 2205 if( aiAppend[i]==0 ){ 2201 2206 aiAppend[i] = p->iLastPg+1; 2202 2207 break; 2203 2208 } 2204 2209 } 2205 2210 }else if( pFS->pCompress==0 ){ ................................................................................ 2222 2227 } 2223 2228 2224 2229 /* 2225 2230 ** Obtain a reference to page number iPg. 2226 2231 ** 2227 2232 ** Return LSM_OK if successful, or an lsm error code if an error occurs. 2228 2233 */ 2229 -int lsmFsDbPageGet(FileSystem *pFS, Segment *pSeg, Pgno iPg, Page **ppPg){ 2234 +int lsmFsDbPageGet(FileSystem *pFS, Segment *pSeg, LsmPgno iPg, Page **ppPg){ 2230 2235 return fsPageGet(pFS, pSeg, iPg, 0, ppPg, 0); 2231 2236 } 2232 2237 2233 2238 /* 2234 2239 ** Obtain a reference to the last page in the segment passed as the 2235 2240 ** second argument. 2236 2241 ** 2237 2242 ** Return LSM_OK if successful, or an lsm error code if an error occurs. 2238 2243 */ 2239 2244 int lsmFsDbPageLast(FileSystem *pFS, Segment *pSeg, Page **ppPg){ 2240 2245 int rc; 2241 - Pgno iPg = pSeg->iLastPg; 2246 + LsmPgno iPg = pSeg->iLastPg; 2242 2247 if( pFS->pCompress ){ 2243 2248 int nSpace; 2244 2249 iPg++; 2245 2250 do { 2246 2251 nSpace = 0; 2247 2252 rc = fsGetPageBefore(pFS, pSeg, iPg, &iPg); 2248 2253 if( rc==LSM_OK ){ ................................................................................ 2362 2367 ** number (*piPg) lies on block iFrom, then calculate the equivalent 2363 2368 ** page on block iTo and set *piPg to this value before returning. 2364 2369 */ 2365 2370 static void fsMovePage( 2366 2371 FileSystem *pFS, /* File system object */ 2367 2372 int iTo, /* Destination block */ 2368 2373 int iFrom, /* Source block */ 2369 - Pgno *piPg /* IN/OUT: Page number */ 2374 + LsmPgno *piPg /* IN/OUT: Page number */ 2370 2375 ){ 2371 - Pgno iPg = *piPg; 2376 + LsmPgno iPg = *piPg; 2372 2377 if( iFrom==fsPageToBlock(pFS, iPg) ){ 2373 2378 const int nPagePerBlock = ( 2374 2379 pFS->pCompress ? pFS ->nBlocksize : (pFS->nBlocksize / pFS->nPagesize) 2375 2380 ); 2376 - *piPg = iPg - (Pgno)(iFrom - iTo) * nPagePerBlock; 2381 + *piPg = iPg - (LsmPgno)(iFrom - iTo) * nPagePerBlock; 2377 2382 } 2378 2383 } 2379 2384 2380 2385 /* 2381 2386 ** Copy the contents of block iFrom to block iTo. 2382 2387 ** 2383 2388 ** It is safe to assume that there are no outstanding references to pages ................................................................................ 2453 2458 /* 2454 2459 ** Append raw data to a segment. Return the database file offset that the 2455 2460 ** data is written to (this may be used as the page number if the data 2456 2461 ** being appended is a new page record). 2457 2462 ** 2458 2463 ** This function is only used in compressed database mode. 2459 2464 */ 2460 -static Pgno fsAppendData( 2465 +static LsmPgno fsAppendData( 2461 2466 FileSystem *pFS, /* File-system handle */ 2462 2467 Segment *pSeg, /* Segment to append to */ 2463 2468 const u8 *aData, /* Buffer containing data to write */ 2464 2469 int nData, /* Size of buffer aData[] in bytes */ 2465 2470 int *pRc /* IN/OUT: Error code */ 2466 2471 ){ 2467 - Pgno iRet = 0; 2472 + LsmPgno iRet = 0; 2468 2473 int rc = *pRc; 2469 2474 assert( pFS->pCompress ); 2470 2475 if( rc==LSM_OK ){ 2471 2476 int nRem = 0; 2472 2477 int nWrite = 0; 2473 - Pgno iLastOnBlock; 2474 - Pgno iApp = pSeg->iLastPg+1; 2478 + LsmPgno iLastOnBlock; 2479 + LsmPgno iApp = pSeg->iLastPg+1; 2475 2480 2476 2481 /* If this is the first data written into the segment, find an append-point 2477 2482 ** or allocate a new block. */ 2478 2483 if( iApp==1 ){ 2479 2484 pSeg->iFirst = iApp = findAppendPoint(pFS, 0); 2480 2485 if( iApp==0 ){ 2481 2486 int iBlk; ................................................................................ 2515 2520 assert( iApp==(fsPageToBlock(pFS, iApp)*pFS->nBlocksize)-4 ); 2516 2521 lsmPutU32(aPtr, iBlk); 2517 2522 rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iApp, aPtr, sizeof(aPtr)); 2518 2523 } 2519 2524 2520 2525 /* Set the "prev" pointer on the new block */ 2521 2526 if( rc==LSM_OK ){ 2522 - Pgno iWrite; 2527 + LsmPgno iWrite; 2523 2528 lsmPutU32(aPtr, fsPageToBlock(pFS, iApp)); 2524 2529 iWrite = fsFirstPageOnBlock(pFS, iBlk); 2525 2530 rc = lsmEnvWrite(pFS->pEnv, pFS->fdDb, iWrite-4, aPtr, sizeof(aPtr)); 2526 2531 if( nRem>0 ) iApp = iWrite; 2527 2532 } 2528 2533 }else{ 2529 2534 /* The next block is already allocated. */ ................................................................................ 2584 2589 ** LSM_OK is returned if successful, or an lsm error code otherwise. If 2585 2590 ** any value other than LSM_OK is returned, then the final value of all 2586 2591 ** output variables is undefined. 2587 2592 */ 2588 2593 static int fsAppendPage( 2589 2594 FileSystem *pFS, 2590 2595 Segment *pSeg, 2591 - Pgno *piNew, 2596 + LsmPgno *piNew, 2592 2597 int *piPrev, 2593 2598 int *piNext 2594 2599 ){ 2595 - Pgno iPrev = pSeg->iLastPg; 2600 + LsmPgno iPrev = pSeg->iLastPg; 2596 2601 int rc; 2597 2602 assert( iPrev!=0 ); 2598 2603 2599 2604 *piPrev = 0; 2600 2605 *piNext = 0; 2601 2606 2602 2607 if( fsIsLast(pFS, iPrev) ){ ................................................................................ 2646 2651 } 2647 2652 *pRc = rc; 2648 2653 } 2649 2654 2650 2655 /* 2651 2656 ** If there exists a hash-table entry associated with page iPg, remove it. 2652 2657 */ 2653 -static void fsRemoveHashEntry(FileSystem *pFS, Pgno iPg){ 2658 +static void fsRemoveHashEntry(FileSystem *pFS, LsmPgno iPg){ 2654 2659 Page *p; 2655 2660 int iHash = fsHashKey(pFS->nHash, iPg); 2656 2661 2657 2662 for(p=pFS->apHash[iHash]; p && p->iPg!=iPg; p=p->pHashNext); 2658 2663 2659 2664 if( p ){ 2660 2665 assert( p->nRef==0 || (p->flags & PAGE_FREE)==0 ); ................................................................................ 2800 2805 int lsmFsSortedPadding( 2801 2806 FileSystem *pFS, 2802 2807 Snapshot *pSnapshot, 2803 2808 Segment *pSeg 2804 2809 ){ 2805 2810 int rc = LSM_OK; 2806 2811 if( pFS->pCompress && pSeg->iFirst ){ 2807 - Pgno iLast2; 2808 - Pgno iLast = pSeg->iLastPg; /* Current last page of segment */ 2812 + LsmPgno iLast2; 2813 + LsmPgno iLast = pSeg->iLastPg; /* Current last page of segment */ 2809 2814 int nPad; /* Bytes of padding required */ 2810 2815 u8 aSz[3]; 2811 2816 2812 2817 iLast2 = (1 + iLast/pFS->szSector) * pFS->szSector - 1; 2813 2818 assert( fsPageToBlock(pFS, iLast)==fsPageToBlock(pFS, iLast2) ); 2814 2819 nPad = (int)(iLast2 - iLast); 2815 2820 ................................................................................ 2931 2936 int lsmFsSectorSize(FileSystem *pFS){ 2932 2937 return pFS->szSector; 2933 2938 } 2934 2939 2935 2940 /* 2936 2941 ** Helper function for lsmInfoArrayStructure(). 2937 2942 */ 2938 -static Segment *startsWith(Segment *pRun, Pgno iFirst){ 2943 +static Segment *startsWith(Segment *pRun, LsmPgno iFirst){ 2939 2944 return (iFirst==pRun->iFirst) ? pRun : 0; 2940 2945 } 2941 2946 2942 2947 /* 2943 2948 ** Return the segment that starts with page iFirst, if any. If no such segment 2944 2949 ** can be found, return NULL. 2945 2950 */ 2946 -static Segment *findSegment(Snapshot *pWorker, Pgno iFirst){ 2951 +static Segment *findSegment(Snapshot *pWorker, LsmPgno iFirst){ 2947 2952 Level *pLvl; /* Used to iterate through db levels */ 2948 2953 Segment *pSeg = 0; /* Pointer to segment to return */ 2949 2954 2950 2955 for(pLvl=lsmDbSnapshotLevel(pWorker); pLvl && pSeg==0; pLvl=pLvl->pNext){ 2951 2956 if( 0==(pSeg = startsWith(&pLvl->lhs, iFirst)) ){ 2952 2957 int i; 2953 2958 for(i=0; i<pLvl->nRight; i++){ ................................................................................ 2966 2971 ** eventually free the string using lsmFree(). 2967 2972 ** 2968 2973 ** If an error occurs, *pzOut is set to NULL and an LSM error code returned. 2969 2974 */ 2970 2975 int lsmInfoArrayStructure( 2971 2976 lsm_db *pDb, 2972 2977 int bBlock, /* True for block numbers only */ 2973 - Pgno iFirst, 2978 + LsmPgno iFirst, 2974 2979 char **pzOut 2975 2980 ){ 2976 2981 int rc = LSM_OK; 2977 2982 Snapshot *pWorker; /* Worker snapshot */ 2978 2983 Segment *pArray = 0; /* Array to report on */ 2979 2984 int bUnlock = 0; 2980 2985 ................................................................................ 3031 3036 } 3032 3037 return rc; 3033 3038 } 3034 3039 3035 3040 int lsmFsSegmentContainsPg( 3036 3041 FileSystem *pFS, 3037 3042 Segment *pSeg, 3038 - Pgno iPg, 3043 + LsmPgno iPg, 3039 3044 int *pbRes 3040 3045 ){ 3041 3046 Redirect *pRedir = pSeg->pRedirect; 3042 3047 int rc = LSM_OK; 3043 3048 int iBlk; 3044 3049 int iLastBlk; 3045 3050 int iPgBlock; /* Block containing page iPg */ ................................................................................ 3060 3065 ** This function implements the lsm_info(LSM_INFO_ARRAY_PAGES) request. 3061 3066 ** If successful, *pzOut is set to point to a nul-terminated string 3062 3067 ** containing the array structure and LSM_OK is returned. The caller should 3063 3068 ** eventually free the string using lsmFree(). 3064 3069 ** 3065 3070 ** If an error occurs, *pzOut is set to NULL and an LSM error code returned. 3066 3071 */ 3067 -int lsmInfoArrayPages(lsm_db *pDb, Pgno iFirst, char **pzOut){ 3072 +int lsmInfoArrayPages(lsm_db *pDb, LsmPgno iFirst, char **pzOut){ 3068 3073 int rc = LSM_OK; 3069 3074 Snapshot *pWorker; /* Worker snapshot */ 3070 3075 Segment *pSeg = 0; /* Array to report on */ 3071 3076 int bUnlock = 0; 3072 3077 3073 3078 *pzOut = 0; 3074 3079 if( iFirst==0 ) return LSM_ERROR; ................................................................................ 3293 3298 #ifndef NDEBUG 3294 3299 /* 3295 3300 ** Return true if pPg happens to be the last page in segment pSeg. Or false 3296 3301 ** otherwise. This function is only invoked as part of assert() conditions. 3297 3302 */ 3298 3303 int lsmFsDbPageIsLast(Segment *pSeg, Page *pPg){ 3299 3304 if( pPg->pFS->pCompress ){ 3300 - Pgno iNext = 0; 3305 + LsmPgno iNext = 0; 3301 3306 int rc; 3302 3307 rc = fsNextPageOffset(pPg->pFS, pSeg, pPg->iPg, pPg->nCompress+6, &iNext); 3303 3308 return (rc!=LSM_OK || iNext==0); 3304 3309 } 3305 3310 return (pPg->iPg==pSeg->iLastPg); 3306 3311 } 3307 3312 #endif
Changes to ext/lsm1/lsm_main.c.
579 579 case LSM_INFO_DB_STRUCTURE: { 580 580 char **pzVal = va_arg(ap, char **); 581 581 rc = lsmStructList(pDb, pzVal); 582 582 break; 583 583 } 584 584 585 585 case LSM_INFO_ARRAY_STRUCTURE: { 586 - Pgno pgno = va_arg(ap, Pgno); 586 + LsmPgno pgno = va_arg(ap, LsmPgno); 587 587 char **pzVal = va_arg(ap, char **); 588 588 rc = lsmInfoArrayStructure(pDb, 0, pgno, pzVal); 589 589 break; 590 590 } 591 591 592 592 case LSM_INFO_ARRAY_PAGES: { 593 - Pgno pgno = va_arg(ap, Pgno); 593 + LsmPgno pgno = va_arg(ap, LsmPgno); 594 594 char **pzVal = va_arg(ap, char **); 595 595 rc = lsmInfoArrayPages(pDb, pgno, pzVal); 596 596 break; 597 597 } 598 598 599 599 case LSM_INFO_PAGE_HEX_DUMP: 600 600 case LSM_INFO_PAGE_ASCII_DUMP: { 601 - Pgno pgno = va_arg(ap, Pgno); 601 + LsmPgno pgno = va_arg(ap, LsmPgno); 602 602 char **pzVal = va_arg(ap, char **); 603 603 int bUnlock = 0; 604 604 rc = infoGetWorker(pDb, 0, &bUnlock); 605 605 if( rc==LSM_OK ){ 606 606 int bHex = (eParam==LSM_INFO_PAGE_HEX_DUMP); 607 607 rc = lsmInfoPageDump(pDb, pgno, bHex, pzVal); 608 608 }
Changes to ext/lsm1/lsm_sorted.c.
100 100 101 101 102 102 #ifndef LSM_SEGMENTPTR_FREE_THRESHOLD 103 103 # define LSM_SEGMENTPTR_FREE_THRESHOLD 1024 104 104 #endif 105 105 106 106 typedef struct SegmentPtr SegmentPtr; 107 -typedef struct Blob Blob; 107 +typedef struct LsmBlob LsmBlob; 108 108 109 -struct Blob { 109 +struct LsmBlob { 110 110 lsm_env *pEnv; 111 111 void *pData; 112 112 int nData; 113 113 int nAlloc; 114 114 }; 115 115 116 116 /* ................................................................................ 125 125 Level *pLevel; /* Level object segment is part of */ 126 126 Segment *pSeg; /* Segment to access */ 127 127 128 128 /* Current page. See segmentPtrLoadPage(). */ 129 129 Page *pPg; /* Current page */ 130 130 u16 flags; /* Copy of page flags field */ 131 131 int nCell; /* Number of cells on pPg */ 132 - Pgno iPtr; /* Base cascade pointer */ 132 + LsmPgno iPtr; /* Base cascade pointer */ 133 133 134 134 /* Current cell. See segmentPtrLoadCell() */ 135 135 int iCell; /* Current record within page pPg */ 136 136 int eType; /* Type of current record */ 137 - Pgno iPgPtr; /* Cascade pointer offset */ 137 + LsmPgno iPgPtr; /* Cascade pointer offset */ 138 138 void *pKey; int nKey; /* Key associated with current record */ 139 139 void *pVal; int nVal; /* Current record value (eType==WRITE only) */ 140 140 141 141 /* Blobs used to allocate buffers for pKey and pVal as required */ 142 - Blob blob1; 143 - Blob blob2; 142 + LsmBlob blob1; 143 + LsmBlob blob2; 144 144 }; 145 145 146 146 /* 147 147 ** Used to iterate through the keys stored in a b-tree hierarchy from start 148 148 ** to finish. Only First() and Next() operations are required. 149 149 ** 150 150 ** btreeCursorNew() ................................................................................ 167 167 int iPg; /* Current entry in aPg[]. -1 -> EOF. */ 168 168 BtreePg *aPg; /* Pages from root to current location */ 169 169 170 170 /* Cache of current entry. pKey==0 for EOF. */ 171 171 void *pKey; 172 172 int nKey; 173 173 int eType; 174 - Pgno iPtr; 174 + LsmPgno iPtr; 175 175 176 176 /* Storage for key, if not local */ 177 - Blob blob; 177 + LsmBlob blob; 178 178 }; 179 179 180 180 181 181 /* 182 182 ** A cursor used for merged searches or iterations through up to one 183 183 ** Tree structure and any number of sorted files. 184 184 ** ................................................................................ 199 199 */ 200 200 struct MultiCursor { 201 201 lsm_db *pDb; /* Connection that owns this cursor */ 202 202 MultiCursor *pNext; /* Next cursor owned by connection pDb */ 203 203 int flags; /* Mask of CURSOR_XXX flags */ 204 204 205 205 int eType; /* Cache of current key type */ 206 - Blob key; /* Cache of current key (or NULL) */ 207 - Blob val; /* Cache of current value */ 206 + LsmBlob key; /* Cache of current key (or NULL) */ 207 + LsmBlob val; /* Cache of current value */ 208 208 209 209 /* All the component cursors: */ 210 210 TreeCursor *apTreeCsr[2]; /* Up to two tree cursors */ 211 211 int iFree; /* Next element of free-list (-ve for eof) */ 212 212 SegmentPtr *aPtr; /* Array of segment pointers */ 213 213 int nPtr; /* Size of array aPtr[] */ 214 214 BtreeCursor *pBtCsr; /* b-tree cursor (db writes only) */ ................................................................................ 217 217 int nTree; /* Size of aTree[] array */ 218 218 int *aTree; /* Array of comparison results */ 219 219 220 220 /* Used by cursors flushing the in-memory tree only */ 221 221 void *pSystemVal; /* Pointer to buffer to free */ 222 222 223 223 /* Used by worker cursors only */ 224 - Pgno *pPrevMergePtr; 224 + LsmPgno *pPrevMergePtr; 225 225 }; 226 226 227 227 /* 228 228 ** The following constants are used to assign integers to each component 229 229 ** cursor of a multi-cursor. 230 230 */ 231 231 #define CURSOR_DATA_TREE0 0 /* Current tree cursor (apTreeCsr[0]) */ ................................................................................ 291 291 lsm_db *pDb; /* Database handle */ 292 292 Level *pLevel; /* Worker snapshot Level being merged */ 293 293 MultiCursor *pCsr; /* Cursor to read new segment contents from */ 294 294 int bFlush; /* True if this is an in-memory tree flush */ 295 295 Hierarchy hier; /* B-tree hierarchy under construction */ 296 296 Page *pPage; /* Current output page */ 297 297 int nWork; /* Number of calls to mergeWorkerNextPage() */ 298 - Pgno *aGobble; /* Gobble point for each input segment */ 298 + LsmPgno *aGobble; /* Gobble point for each input segment */ 299 299 300 - Pgno iIndirect; 300 + LsmPgno iIndirect; 301 301 struct SavedPgno { 302 - Pgno iPgno; 302 + LsmPgno iPgno; 303 303 int bStore; 304 304 } aSave[2]; 305 305 }; 306 306 307 307 #ifdef LSM_DEBUG_EXPENSIVE 308 308 static int assertPointersOk(lsm_db *, Segment *, Segment *, int); 309 309 static int assertBtreeOk(lsm_db *, Segment *); ................................................................................ 367 367 aOut[3] = (u8)((nVal>>32) & 0xFF); 368 368 aOut[4] = (u8)((nVal>>24) & 0xFF); 369 369 aOut[5] = (u8)((nVal>>16) & 0xFF); 370 370 aOut[6] = (u8)((nVal>> 8) & 0xFF); 371 371 aOut[7] = (u8)((nVal ) & 0xFF); 372 372 } 373 373 374 -static int sortedBlobGrow(lsm_env *pEnv, Blob *pBlob, int nData){ 374 +static int sortedBlobGrow(lsm_env *pEnv, LsmBlob *pBlob, int nData){ 375 375 assert( pBlob->pEnv==pEnv || (pBlob->pEnv==0 && pBlob->pData==0) ); 376 376 if( pBlob->nAlloc<nData ){ 377 377 pBlob->pData = lsmReallocOrFree(pEnv, pBlob->pData, nData); 378 378 if( !pBlob->pData ) return LSM_NOMEM_BKPT; 379 379 pBlob->nAlloc = nData; 380 380 pBlob->pEnv = pEnv; 381 381 } 382 382 return LSM_OK; 383 383 } 384 384 385 -static int sortedBlobSet(lsm_env *pEnv, Blob *pBlob, void *pData, int nData){ 385 +static int sortedBlobSet(lsm_env *pEnv, LsmBlob *pBlob, void *pData, int nData){ 386 386 if( sortedBlobGrow(pEnv, pBlob, nData) ) return LSM_NOMEM; 387 387 memcpy(pBlob->pData, pData, nData); 388 388 pBlob->nData = nData; 389 389 return LSM_OK; 390 390 } 391 391 392 392 #if 0 393 -static int sortedBlobCopy(Blob *pDest, Blob *pSrc){ 393 +static int sortedBlobCopy(LsmBlob *pDest, LsmBlob *pSrc){ 394 394 return sortedBlobSet(pDest, pSrc->pData, pSrc->nData); 395 395 } 396 396 #endif 397 397 398 -static void sortedBlobFree(Blob *pBlob){ 398 +static void sortedBlobFree(LsmBlob *pBlob){ 399 399 assert( pBlob->pEnv || pBlob->pData==0 ); 400 400 if( pBlob->pData ) lsmFree(pBlob->pEnv, pBlob->pData); 401 - memset(pBlob, 0, sizeof(Blob)); 401 + memset(pBlob, 0, sizeof(LsmBlob)); 402 402 } 403 403 404 404 static int sortedReadData( 405 405 Segment *pSeg, 406 406 Page *pPg, 407 407 int iOff, 408 408 int nByte, 409 409 void **ppData, 410 - Blob *pBlob 410 + LsmBlob *pBlob 411 411 ){ 412 412 int rc = LSM_OK; 413 413 int iEnd; 414 414 int nData; 415 415 int nCell; 416 416 u8 *aData; 417 417 ................................................................................ 477 477 return rc; 478 478 } 479 479 480 480 static int pageGetNRec(u8 *aData, int nData){ 481 481 return (int)lsmGetU16(&aData[SEGMENT_NRECORD_OFFSET(nData)]); 482 482 } 483 483 484 -static Pgno pageGetPtr(u8 *aData, int nData){ 485 - return (Pgno)lsmGetU64(&aData[SEGMENT_POINTER_OFFSET(nData)]); 484 +static LsmPgno pageGetPtr(u8 *aData, int nData){ 485 + return (LsmPgno)lsmGetU64(&aData[SEGMENT_POINTER_OFFSET(nData)]); 486 486 } 487 487 488 488 static int pageGetFlags(u8 *aData, int nData){ 489 489 return (int)lsmGetU16(&aData[SEGMENT_FLAGS_OFFSET(nData)]); 490 490 } 491 491 492 492 static u8 *pageGetCell(u8 *aData, int nData, int iCell){ ................................................................................ 502 502 return pageGetNRec(aData, nData); 503 503 } 504 504 505 505 /* 506 506 ** Return the decoded (possibly relative) pointer value stored in cell 507 507 ** iCell from page aData/nData. 508 508 */ 509 -static Pgno pageGetRecordPtr(u8 *aData, int nData, int iCell){ 510 - Pgno iRet; /* Return value */ 509 +static LsmPgno pageGetRecordPtr(u8 *aData, int nData, int iCell){ 510 + LsmPgno iRet; /* Return value */ 511 511 u8 *aCell; /* Pointer to cell iCell */ 512 512 513 513 assert( iCell<pageGetNRec(aData, nData) && iCell>=0 ); 514 514 aCell = pageGetCell(aData, nData, iCell); 515 515 lsmVarintGet64(&aCell[1], &iRet); 516 516 return iRet; 517 517 } ................................................................................ 518 518 519 519 static u8 *pageGetKey( 520 520 Segment *pSeg, /* Segment pPg belongs to */ 521 521 Page *pPg, /* Page to read from */ 522 522 int iCell, /* Index of cell on page to read */ 523 523 int *piTopic, /* OUT: Topic associated with this key */ 524 524 int *pnKey, /* OUT: Size of key in bytes */ 525 - Blob *pBlob /* If required, use this for dynamic memory */ 525 + LsmBlob *pBlob /* If required, use this for dynamic memory */ 526 526 ){ 527 527 u8 *pKey; 528 528 int nDummy; 529 529 int eType; 530 530 u8 *aData; 531 531 int nData; 532 532 ................................................................................ 550 550 551 551 static int pageGetKeyCopy( 552 552 lsm_env *pEnv, /* Environment handle */ 553 553 Segment *pSeg, /* Segment pPg belongs to */ 554 554 Page *pPg, /* Page to read from */ 555 555 int iCell, /* Index of cell on page to read */ 556 556 int *piTopic, /* OUT: Topic associated with this key */ 557 - Blob *pBlob /* If required, use this for dynamic memory */ 557 + LsmBlob *pBlob /* If required, use this for dynamic memory */ 558 558 ){ 559 559 int rc = LSM_OK; 560 560 int nKey; 561 561 u8 *aKey; 562 562 563 563 aKey = pageGetKey(pSeg, pPg, iCell, piTopic, &nKey, pBlob); 564 564 assert( (void *)aKey!=pBlob->pData || nKey==pBlob->nData ); ................................................................................ 565 565 if( (void *)aKey!=pBlob->pData ){ 566 566 rc = sortedBlobSet(pEnv, pBlob, aKey, nKey); 567 567 } 568 568 569 569 return rc; 570 570 } 571 571 572 -static Pgno pageGetBtreeRef(Page *pPg, int iKey){ 573 - Pgno iRef; 572 +static LsmPgno pageGetBtreeRef(Page *pPg, int iKey){ 573 + LsmPgno iRef; 574 574 u8 *aData; 575 575 int nData; 576 576 u8 *aCell; 577 577 578 578 aData = fsPageData(pPg, &nData); 579 579 aCell = pageGetCell(aData, nData, iKey); 580 580 assert( aCell[0]==0 ); ................................................................................ 588 588 #define GETVARINT64(a, i) (((i)=((u8*)(a))[0])<=240?1:lsmVarintGet64((a), &(i))) 589 589 #define GETVARINT32(a, i) (((i)=((u8*)(a))[0])<=240?1:lsmVarintGet32((a), &(i))) 590 590 591 591 static int pageGetBtreeKey( 592 592 Segment *pSeg, /* Segment page pPg belongs to */ 593 593 Page *pPg, 594 594 int iKey, 595 - Pgno *piPtr, 595 + LsmPgno *piPtr, 596 596 int *piTopic, 597 597 void **ppKey, 598 598 int *pnKey, 599 - Blob *pBlob 599 + LsmBlob *pBlob 600 600 ){ 601 601 u8 *aData; 602 602 int nData; 603 603 u8 *aCell; 604 604 int eType; 605 605 606 606 aData = fsPageData(pPg, &nData); ................................................................................ 609 609 610 610 aCell = pageGetCell(aData, nData, iKey); 611 611 eType = *aCell++; 612 612 aCell += GETVARINT64(aCell, *piPtr); 613 613 614 614 if( eType==0 ){ 615 615 int rc; 616 - Pgno iRef; /* Page number of referenced page */ 616 + LsmPgno iRef; /* Page number of referenced page */ 617 617 Page *pRef; 618 618 aCell += GETVARINT64(aCell, iRef); 619 619 rc = lsmFsDbPageGet(lsmPageFS(pPg), pSeg, iRef, &pRef); 620 620 if( rc!=LSM_OK ) return rc; 621 621 pageGetKeyCopy(lsmPageEnv(pPg), pSeg, pRef, 0, &eType, pBlob); 622 622 lsmFsPageRelease(pRef); 623 623 *ppKey = pBlob->pData; ................................................................................ 634 634 static int btreeCursorLoadKey(BtreeCursor *pCsr){ 635 635 int rc = LSM_OK; 636 636 if( pCsr->iPg<0 ){ 637 637 pCsr->pKey = 0; 638 638 pCsr->nKey = 0; 639 639 pCsr->eType = 0; 640 640 }else{ 641 - Pgno dummy; 641 + LsmPgno dummy; 642 642 int iPg = pCsr->iPg; 643 643 int iCell = pCsr->aPg[iPg].iCell; 644 644 while( iCell<0 && (--iPg)>=0 ){ 645 645 iCell = pCsr->aPg[iPg].iCell-1; 646 646 } 647 647 if( iPg<0 || iCell<0 ) return LSM_CORRUPT_BKPT; 648 648 ................................................................................ 679 679 assert( pCsr->iPg==pCsr->nDepth-1 ); 680 680 681 681 aData = fsPageData(pPg->pPage, &nData); 682 682 nCell = pageGetNRec(aData, nData); 683 683 assert( pPg->iCell<=nCell ); 684 684 pPg->iCell++; 685 685 if( pPg->iCell==nCell ){ 686 - Pgno iLoad; 686 + LsmPgno iLoad; 687 687 688 688 /* Up to parent. */ 689 689 lsmFsPageRelease(pPg->pPage); 690 690 pPg->pPage = 0; 691 691 pCsr->iPg--; 692 692 while( pCsr->iPg>=0 ){ 693 693 pPg = &pCsr->aPg[pCsr->iPg]; ................................................................................ 838 838 MergeInput *p 839 839 ){ 840 840 int rc = LSM_OK; 841 841 842 842 if( p->iPg ){ 843 843 lsm_env *pEnv = lsmFsEnv(pCsr->pFS); 844 844 int iCell; /* Current cell number on leaf page */ 845 - Pgno iLeaf; /* Page number of current leaf page */ 845 + LsmPgno iLeaf; /* Page number of current leaf page */ 846 846 int nDepth; /* Depth of b-tree structure */ 847 847 Segment *pSeg = pCsr->pSeg; 848 848 849 849 /* Decode the MergeInput structure */ 850 850 iLeaf = p->iPg; 851 851 nDepth = (p->iCell & 0x00FF); 852 852 iCell = (p->iCell >> 8) - 1; ................................................................................ 862 862 pCsr->nDepth = nDepth; 863 863 pCsr->aPg[pCsr->iPg].iCell = iCell; 864 864 rc = lsmFsDbPageGet(pCsr->pFS, pSeg, iLeaf, pp); 865 865 } 866 866 867 867 /* Populate any other aPg[] array entries */ 868 868 if( rc==LSM_OK && nDepth>1 ){ 869 - Blob blob = {0,0,0}; 869 + LsmBlob blob = {0,0,0}; 870 870 void *pSeek; 871 871 int nSeek; 872 872 int iTopicSeek; 873 873 int iPg = 0; 874 874 int iLoad = (int)pSeg->iRoot; 875 875 Page *pPg = pCsr->aPg[nDepth-1].pPage; 876 876 ................................................................................ 879 879 ** In this case, set the iTopicSeek/pSeek/nSeek key to a value 880 880 ** greater than any real key. */ 881 881 assert( iCell==-1 ); 882 882 iTopicSeek = 1000; 883 883 pSeek = 0; 884 884 nSeek = 0; 885 885 }else{ 886 - Pgno dummy; 886 + LsmPgno dummy; 887 887 rc = pageGetBtreeKey(pSeg, pPg, 888 888 0, &dummy, &iTopicSeek, &pSeek, &nSeek, &pCsr->blob 889 889 ); 890 890 } 891 891 892 892 do { 893 893 Page *pPg2; ................................................................................ 908 908 iMax = iCell2-1; 909 909 iMin = 0; 910 910 911 911 while( iMax>=iMin ){ 912 912 int iTry = (iMin+iMax)/2; 913 913 void *pKey; int nKey; /* Key for cell iTry */ 914 914 int iTopic; /* Topic for key pKeyT/nKeyT */ 915 - Pgno iPtr; /* Pointer for cell iTry */ 915 + LsmPgno iPtr; /* Pointer for cell iTry */ 916 916 int res; /* (pSeek - pKeyT) */ 917 917 918 918 rc = pageGetBtreeKey( 919 919 pSeg, pPg2, iTry, &iPtr, &iTopic, &pKey, &nKey, &blob 920 920 ); 921 921 if( rc!=LSM_OK ) break; 922 922 ................................................................................ 951 951 u8 *aData; 952 952 int nData; 953 953 954 954 pBtreePg = &pCsr->aPg[pCsr->iPg]; 955 955 aData = fsPageData(pBtreePg->pPage, &nData); 956 956 pCsr->iPtr = btreeCursorPtr(aData, nData, pBtreePg->iCell+1); 957 957 if( pBtreePg->iCell<0 ){ 958 - Pgno dummy; 958 + LsmPgno dummy; 959 959 int i; 960 960 for(i=pCsr->iPg-1; i>=0; i--){ 961 961 if( pCsr->aPg[i].iCell>0 ) break; 962 962 } 963 963 assert( i>=0 ); 964 964 rc = pageGetBtreeKey(pSeg, 965 965 pCsr->aPg[i].pPage, pCsr->aPg[i].iCell-1, ................................................................................ 1026 1026 } 1027 1027 1028 1028 static int segmentPtrReadData( 1029 1029 SegmentPtr *pPtr, 1030 1030 int iOff, 1031 1031 int nByte, 1032 1032 void **ppData, 1033 - Blob *pBlob 1033 + LsmBlob *pBlob 1034 1034 ){ 1035 1035 return sortedReadData(pPtr->pSeg, pPtr->pPg, iOff, nByte, ppData, pBlob); 1036 1036 } 1037 1037 1038 1038 static int segmentPtrNextPage( 1039 1039 SegmentPtr *pPtr, /* Load page into this SegmentPtr object */ 1040 1040 int eDir /* +1 for next(), -1 for prev() */ ................................................................................ 1119 1119 1120 1120 pSeg = sortedSplitkeySegment(pLevel); 1121 1121 if( rc==LSM_OK ){ 1122 1122 rc = lsmFsDbPageGet(pDb->pFS, pSeg, pMerge->splitkey.iPg, &pPg); 1123 1123 } 1124 1124 if( rc==LSM_OK ){ 1125 1125 int iTopic; 1126 - Blob blob = {0, 0, 0, 0}; 1126 + LsmBlob blob = {0, 0, 0, 0}; 1127 1127 u8 *aData; 1128 1128 int nData; 1129 1129 1130 1130 aData = lsmFsPageData(pPg, &nData); 1131 1131 if( pageGetFlags(aData, nData) & SEGMENT_BTREE_FLAG ){ 1132 1132 void *pKey; 1133 1133 int nKey; 1134 - Pgno dummy; 1134 + LsmPgno dummy; 1135 1135 rc = pageGetBtreeKey(pSeg, 1136 1136 pPg, pMerge->splitkey.iCell, &dummy, &iTopic, &pKey, &nKey, &blob 1137 1137 ); 1138 1138 if( rc==LSM_OK && blob.pData!=pKey ){ 1139 1139 rc = sortedBlobSet(pEnv, &blob, pKey, nKey); 1140 1140 } 1141 1141 }else{ ................................................................................ 1338 1338 */ 1339 1339 static int assertKeyLocation( 1340 1340 MultiCursor *pCsr, 1341 1341 SegmentPtr *pPtr, 1342 1342 void *pKey, int nKey 1343 1343 ){ 1344 1344 lsm_env *pEnv = lsmFsEnv(pCsr->pDb->pFS); 1345 - Blob blob = {0, 0, 0}; 1345 + LsmBlob blob = {0, 0, 0}; 1346 1346 int eDir; 1347 1347 int iTopic = 0; /* TODO: Fix me */ 1348 1348 1349 1349 for(eDir=-1; eDir<=1; eDir+=2){ 1350 1350 Page *pTest = pPtr->pPg; 1351 1351 1352 1352 lsmFsPageRef(pTest); ................................................................................ 1484 1484 return rc; 1485 1485 } 1486 1486 1487 1487 static int ptrFwdPointer( 1488 1488 Page *pPage, 1489 1489 int iCell, 1490 1490 Segment *pSeg, 1491 - Pgno *piPtr, 1491 + LsmPgno *piPtr, 1492 1492 int *pbFound 1493 1493 ){ 1494 1494 Page *pPg = pPage; 1495 1495 int iFirst = iCell; 1496 1496 int rc = LSM_OK; 1497 1497 1498 1498 do { ................................................................................ 1569 1569 ** much better if the multi-cursor could do this lazily - only seek to the 1570 1570 ** level (N+1) page after the user has moved the cursor on level N passed 1571 1571 ** the big range-delete. 1572 1572 */ 1573 1573 static int segmentPtrFwdPointer( 1574 1574 MultiCursor *pCsr, /* Multi-cursor pPtr belongs to */ 1575 1575 SegmentPtr *pPtr, /* Segment-pointer to extract FC ptr from */ 1576 - Pgno *piPtr /* OUT: FC pointer value */ 1576 + LsmPgno *piPtr /* OUT: FC pointer value */ 1577 1577 ){ 1578 1578 Level *pLvl = pPtr->pLevel; 1579 1579 Level *pNext = pLvl->pNext; 1580 1580 Page *pPg = pPtr->pPg; 1581 1581 int rc; 1582 1582 int bFound; 1583 - Pgno iOut = 0; 1583 + LsmPgno iOut = 0; 1584 1584 1585 1585 if( pPtr->pSeg==&pLvl->lhs || pPtr->pSeg==&pLvl->aRhs[pLvl->nRight-1] ){ 1586 1586 if( pNext==0 1587 1587 || (pNext->nRight==0 && pNext->lhs.iRoot) 1588 1588 || (pNext->nRight!=0 && pNext->aRhs[0].iRoot) 1589 1589 ){ 1590 1590 /* Do nothing. The pointer will not be used anyway. */ ................................................................................ 1637 1637 int *pbStop 1638 1638 ){ 1639 1639 int (*xCmp)(void *, int, void *, int) = pCsr->pDb->xCmp; 1640 1640 int res = 0; /* Result of comparison operation */ 1641 1641 int rc = LSM_OK; 1642 1642 int iMin; 1643 1643 int iMax; 1644 - Pgno iPtrOut = 0; 1644 + LsmPgno iPtrOut = 0; 1645 1645 1646 1646 /* If the current page contains an oversized entry, then there are no 1647 1647 ** pointers to one or more of the subsequent pages in the sorted run. 1648 1648 ** The following call ensures that the segment-ptr points to the correct 1649 1649 ** page in this case. */ 1650 1650 rc = segmentPtrSearchOversized(pCsr, pPtr, iTopic, pKey, nKey); 1651 1651 iPtrOut = pPtr->iPtr; ................................................................................ 1764 1764 } 1765 1765 1766 1766 static int seekInBtree( 1767 1767 MultiCursor *pCsr, /* Multi-cursor object */ 1768 1768 Segment *pSeg, /* Seek within this segment */ 1769 1769 int iTopic, 1770 1770 void *pKey, int nKey, /* Key to seek to */ 1771 - Pgno *aPg, /* OUT: Page numbers */ 1771 + LsmPgno *aPg, /* OUT: Page numbers */ 1772 1772 Page **ppPg /* OUT: Leaf (sorted-run) page reference */ 1773 1773 ){ 1774 1774 int i = 0; 1775 1775 int rc; 1776 1776 int iPg; 1777 1777 Page *pPg = 0; 1778 - Blob blob = {0, 0, 0}; 1778 + LsmBlob blob = {0, 0, 0}; 1779 1779 1780 1780 iPg = (int)pSeg->iRoot; 1781 1781 do { 1782 - Pgno *piFirst = 0; 1782 + LsmPgno *piFirst = 0; 1783 1783 if( aPg ){ 1784 1784 aPg[i++] = iPg; 1785 1785 piFirst = &aPg[i]; 1786 1786 } 1787 1787 1788 1788 rc = lsmFsDbPageGet(pCsr->pDb->pFS, pSeg, iPg, &pPg); 1789 1789 assert( rc==LSM_OK || pPg==0 ); ................................................................................ 1804 1804 1805 1805 iMin = 0; 1806 1806 iMax = nRec-1; 1807 1807 while( iMax>=iMin ){ 1808 1808 int iTry = (iMin+iMax)/2; 1809 1809 void *pKeyT; int nKeyT; /* Key for cell iTry */ 1810 1810 int iTopicT; /* Topic for key pKeyT/nKeyT */ 1811 - Pgno iPtr; /* Pointer associated with cell iTry */ 1811 + LsmPgno iPtr; /* Pointer associated with cell iTry */ 1812 1812 int res; /* (pKey - pKeyT) */ 1813 1813 1814 1814 rc = pageGetBtreeKey( 1815 1815 pSeg, pPg, iTry, &iPtr, &iTopicT, &pKeyT, &nKeyT, &blob 1816 1816 ); 1817 1817 if( rc!=LSM_OK ) break; 1818 1818 if( piFirst && pKeyT==blob.pData ){ ................................................................................ 1895 1895 */ 1896 1896 static int seekInLevel( 1897 1897 MultiCursor *pCsr, /* Sorted cursor object to seek */ 1898 1898 SegmentPtr *aPtr, /* Pointer to array of (nRhs+1) SPs */ 1899 1899 int eSeek, /* Search bias - see above */ 1900 1900 int iTopic, /* Key topic to search for */ 1901 1901 void *pKey, int nKey, /* Key to search for */ 1902 - Pgno *piPgno, /* IN/OUT: fraction cascade pointer (or 0) */ 1902 + LsmPgno *piPgno, /* IN/OUT: fraction cascade pointer (or 0) */ 1903 1903 int *pbStop /* OUT: See above */ 1904 1904 ){ 1905 1905 Level *pLvl = aPtr[0].pLevel; /* Level to seek within */ 1906 1906 int rc = LSM_OK; /* Return code */ 1907 1907 int iOut = 0; /* Pointer to return to caller */ 1908 1908 int res = -1; /* Result of xCmp(pKey, split) */ 1909 1909 int nRhs = pLvl->nRight; /* Number of right-hand-side segments */ ................................................................................ 3051 3051 void *pKey, int nKey, 3052 3052 int eSeek 3053 3053 ){ 3054 3054 int eESeek = eSeek; /* Effective eSeek parameter */ 3055 3055 int bStop = 0; /* Set to true to halt search operation */ 3056 3056 int rc = LSM_OK; /* Return code */ 3057 3057 int iPtr = 0; /* Used to iterate through pCsr->aPtr[] */ 3058 - Pgno iPgno = 0; /* FC pointer value */ 3058 + LsmPgno iPgno = 0; /* FC pointer value */ 3059 3059 3060 3060 assert( pCsr->apTreeCsr[0]==0 || iTopic==0 ); 3061 3061 assert( pCsr->apTreeCsr[1]==0 || iTopic==0 ); 3062 3062 3063 3063 if( eESeek==LSM_SEEK_LEFAST ) eESeek = LSM_SEEK_LE; 3064 3064 3065 3065 assert( eESeek==LSM_SEEK_EQ || eESeek==LSM_SEEK_LE || eESeek==LSM_SEEK_GE ); ................................................................................ 3533 3533 ** differences are: 3534 3534 ** 3535 3535 ** 1. The record format is (usually, see below) as follows: 3536 3536 ** 3537 3537 ** + Type byte (always SORTED_SEPARATOR or SORTED_SYSTEM_SEPARATOR), 3538 3538 ** + Absolute pointer value (varint), 3539 3539 ** + Number of bytes in key (varint), 3540 -** + Blob containing key data. 3540 +** + LsmBlob containing key data. 3541 3541 ** 3542 3542 ** 2. All pointer values are stored as absolute values (not offsets 3543 3543 ** relative to the footer pointer value). 3544 3544 ** 3545 3545 ** 3. Each pointer that is part of a record points to a page that 3546 3546 ** contains keys smaller than the records key (note: not "equal to or 3547 3547 ** smaller than - smaller than"). ................................................................................ 3567 3567 ** 3568 3568 ** See function seekInBtree() for the code that traverses b-tree pages. 3569 3569 */ 3570 3570 3571 3571 static int mergeWorkerBtreeWrite( 3572 3572 MergeWorker *pMW, 3573 3573 u8 eType, 3574 - Pgno iPtr, 3575 - Pgno iKeyPg, 3574 + LsmPgno iPtr, 3575 + LsmPgno iKeyPg, 3576 3576 void *pKey, 3577 3577 int nKey 3578 3578 ){ 3579 3579 Hierarchy *p = &pMW->hier; 3580 3580 lsm_db *pDb = pMW->pDb; /* Database handle */ 3581 3581 int rc = LSM_OK; /* Return Code */ 3582 3582 int iLevel; /* Level of b-tree hierachy to write to */ ................................................................................ 3678 3678 3679 3679 return rc; 3680 3680 } 3681 3681 3682 3682 static int mergeWorkerBtreeIndirect(MergeWorker *pMW){ 3683 3683 int rc = LSM_OK; 3684 3684 if( pMW->iIndirect ){ 3685 - Pgno iKeyPg = pMW->aSave[1].iPgno; 3685 + LsmPgno iKeyPg = pMW->aSave[1].iPgno; 3686 3686 rc = mergeWorkerBtreeWrite(pMW, 0, pMW->iIndirect, iKeyPg, 0, 0); 3687 3687 pMW->iIndirect = 0; 3688 3688 } 3689 3689 return rc; 3690 3690 } 3691 3691 3692 3692 /* ................................................................................ 3699 3699 static int mergeWorkerPushHierarchy( 3700 3700 MergeWorker *pMW, /* Merge worker object */ 3701 3701 int iTopic, /* Topic value for this key */ 3702 3702 void *pKey, /* Pointer to key buffer */ 3703 3703 int nKey /* Size of pKey buffer in bytes */ 3704 3704 ){ 3705 3705 int rc = LSM_OK; /* Return Code */ 3706 - Pgno iPtr; /* Pointer value to accompany pKey/nKey */ 3706 + LsmPgno iPtr; /* Pointer value to accompany pKey/nKey */ 3707 3707 3708 3708 assert( pMW->aSave[0].bStore==0 ); 3709 3709 assert( pMW->aSave[1].bStore==0 ); 3710 3710 rc = mergeWorkerBtreeIndirect(pMW); 3711 3711 3712 3712 /* Obtain the absolute pointer value to store along with the key in the 3713 3713 ** page body. This pointer points to a page that contains keys that are ................................................................................ 3730 3730 } 3731 3731 3732 3732 static int mergeWorkerFinishHierarchy( 3733 3733 MergeWorker *pMW /* Merge worker object */ 3734 3734 ){ 3735 3735 int i; /* Used to loop through apHier[] */ 3736 3736 int rc = LSM_OK; /* Return code */ 3737 - Pgno iPtr; /* New right-hand-child pointer value */ 3737 + LsmPgno iPtr; /* New right-hand-child pointer value */ 3738 3738 3739 3739 iPtr = pMW->aSave[0].iPgno; 3740 3740 for(i=0; i<pMW->hier.nHier && rc==LSM_OK; i++){ 3741 3741 Page *pPg = pMW->hier.apHier[i]; 3742 3742 int nData; /* Size of aData[] in bytes */ 3743 3743 u8 *aData; /* Page data for pPg */ 3744 3744 ................................................................................ 3826 3826 ** zero records. The flags field is cleared. The page footer pointer field 3827 3827 ** is set to iFPtr. 3828 3828 ** 3829 3829 ** If successful, LSM_OK is returned. Otherwise, an error code. 3830 3830 */ 3831 3831 static int mergeWorkerNextPage( 3832 3832 MergeWorker *pMW, /* Merge worker object to append page to */ 3833 - Pgno iFPtr /* Pointer value for footer of new page */ 3833 + LsmPgno iFPtr /* Pointer value for footer of new page */ 3834 3834 ){ 3835 3835 int rc = LSM_OK; /* Return code */ 3836 3836 Page *pNext = 0; /* New page appended to run */ 3837 3837 lsm_db *pDb = pMW->pDb; /* Database handle */ 3838 3838 3839 3839 rc = lsmFsSortedAppend(pDb->pFS, pDb->pWorker, pMW->pLevel, 0, &pNext); 3840 3840 assert( rc || pMW->pLevel->lhs.iFirst>0 || pMW->pDb->compress.xCompress ); ................................................................................ 4214 4214 4215 4215 static int mergeWorkerStep(MergeWorker *pMW){ 4216 4216 lsm_db *pDb = pMW->pDb; /* Database handle */ 4217 4217 MultiCursor *pCsr; /* Cursor to read input data from */ 4218 4218 int rc = LSM_OK; /* Return code */ 4219 4219 int eType; /* SORTED_SEPARATOR, WRITE or DELETE */ 4220 4220 void *pKey; int nKey; /* Key */ 4221 - Pgno iPtr; 4221 + LsmPgno iPtr; 4222 4222 int iVal; 4223 4223 4224 4224 pCsr = pMW->pCsr; 4225 4225 4226 4226 /* Pull the next record out of the source cursor. */ 4227 4227 lsmMCursorKey(pCsr, &pKey, &nKey); 4228 4228 eType = pCsr->eType; ................................................................................ 4367 4367 multiCursorIgnoreDelete(pCsr); 4368 4368 } 4369 4369 } 4370 4370 4371 4371 if( rc!=LSM_OK ){ 4372 4372 lsmMCursorClose(pCsr, 0); 4373 4373 }else{ 4374 - Pgno iLeftPtr = 0; 4374 + LsmPgno iLeftPtr = 0; 4375 4375 Merge merge; /* Merge object used to create new level */ 4376 4376 MergeWorker mergeworker; /* MergeWorker object for the same purpose */ 4377 4377 4378 4378 memset(&merge, 0, sizeof(Merge)); 4379 4379 memset(&mergeworker, 0, sizeof(MergeWorker)); 4380 4380 4381 4381 pNew->pMerge = &merge; ................................................................................ 4544 4544 assert( pDb->pWorker ); 4545 4545 assert( pLevel->pMerge ); 4546 4546 assert( pLevel->nRight>0 ); 4547 4547 4548 4548 memset(pMW, 0, sizeof(MergeWorker)); 4549 4549 pMW->pDb = pDb; 4550 4550 pMW->pLevel = pLevel; 4551 - pMW->aGobble = lsmMallocZeroRc(pDb->pEnv, sizeof(Pgno) * pLevel->nRight, &rc); 4551 + pMW->aGobble = lsmMallocZeroRc(pDb->pEnv, sizeof(LsmPgno)*pLevel->nRight,&rc); 4552 4552 4553 4553 /* Create a multi-cursor to read the data to write to the new 4554 4554 ** segment. The new segment contains: 4555 4555 ** 4556 4556 ** 1. Records from LHS of each of the nMerge levels being merged. 4557 4557 ** 2. Separators from either the last level being merged, or the 4558 4558 ** separators attached to the LHS of the following level, or neither. ................................................................................ 4626 4626 lsm_db *pDb, /* Worker connection */ 4627 4627 MultiCursor *pCsr, /* Multi-cursor being used for a merge */ 4628 4628 int iGobble /* pCsr->aPtr[] entry to operate on */ 4629 4629 ){ 4630 4630 int rc = LSM_OK; 4631 4631 if( rtTopic(pCsr->eType)==0 ){ 4632 4632 Segment *pSeg = pCsr->aPtr[iGobble].pSeg; 4633 - Pgno *aPg; 4633 + LsmPgno *aPg; 4634 4634 int nPg; 4635 4635 4636 4636 /* Seek from the root of the b-tree to the segment leaf that may contain 4637 4637 ** a key equal to the one multi-cursor currently points to. Record the 4638 4638 ** page number of each b-tree page and the leaf. The segment may be 4639 4639 ** gobbled up to (but not including) the first of these page numbers. 4640 4640 */ 4641 4641 assert( pSeg->iRoot>0 ); 4642 - aPg = lsmMallocZeroRc(pDb->pEnv, sizeof(Pgno)*32, &rc); 4642 + aPg = lsmMallocZeroRc(pDb->pEnv, sizeof(LsmPgno)*32, &rc); 4643 4643 if( rc==LSM_OK ){ 4644 4644 rc = seekInBtree(pCsr, pSeg, 4645 4645 rtTopic(pCsr->eType), pCsr->key.pData, pCsr->key.nData, aPg, 0 4646 4646 ); 4647 4647 } 4648 4648 4649 4649 if( rc==LSM_OK ){ ................................................................................ 5462 5462 /* 5463 5463 ** Return a string representation of the segment passed as the only argument. 5464 5464 ** Space for the returned string is allocated using lsmMalloc(), and should 5465 5465 ** be freed by the caller using lsmFree(). 5466 5466 */ 5467 5467 static char *segToString(lsm_env *pEnv, Segment *pSeg, int nMin){ 5468 5468 int nSize = pSeg->nSize; 5469 - Pgno iRoot = pSeg->iRoot; 5470 - Pgno iFirst = pSeg->iFirst; 5471 - Pgno iLast = pSeg->iLastPg; 5469 + LsmPgno iRoot = pSeg->iRoot; 5470 + LsmPgno iFirst = pSeg->iFirst; 5471 + LsmPgno iLast = pSeg->iLastPg; 5472 5472 char *z; 5473 5473 5474 5474 char *z1; 5475 5475 char *z2; 5476 5476 int nPad; 5477 5477 5478 5478 z1 = lsmMallocPrintf(pEnv, "%d.%d", iFirst, iLast); ................................................................................ 5523 5523 aBuf[0] = '\0'; 5524 5524 } 5525 5525 5526 5526 return i; 5527 5527 } 5528 5528 5529 5529 void sortedDumpPage(lsm_db *pDb, Segment *pRun, Page *pPg, int bVals){ 5530 - Blob blob = {0, 0, 0}; /* Blob used for keys */ 5530 + LsmBlob blob = {0, 0, 0}; /* LsmBlob used for keys */ 5531 5531 LsmString s; 5532 5532 int i; 5533 5533 5534 5534 int nRec; 5535 5535 int iPtr; 5536 5536 int flags; 5537 5537 u8 *aData; ................................................................................ 5559 5559 5560 5560 aCell = pageGetCell(aData, nData, i); 5561 5561 eType = *aCell++; 5562 5562 assert( (flags & SEGMENT_BTREE_FLAG) || eType!=0 ); 5563 5563 aCell += lsmVarintGet32(aCell, &iPgPtr); 5564 5564 5565 5565 if( eType==0 ){ 5566 - Pgno iRef; /* Page number of referenced page */ 5566 + LsmPgno iRef; /* Page number of referenced page */ 5567 5567 aCell += lsmVarintGet64(aCell, &iRef); 5568 5568 lsmFsDbPageGet(pDb->pFS, pRun, iRef, &pRef); 5569 5569 aKey = pageGetKey(pRun, pRef, 0, &iTopic, &nKey, &blob); 5570 5570 }else{ 5571 5571 aCell += lsmVarintGet32(aCell, &nKey); 5572 5572 if( rtIsWrite(eType) ) aCell += lsmVarintGet32(aCell, &nVal); 5573 5573 sortedReadData(0, pPg, (aCell-aData), nKey+nVal, (void **)&aKey, &blob); ................................................................................ 5603 5603 int bIndirect, /* True to follow indirect refs */ 5604 5604 Page *pPg, 5605 5605 int iCell, 5606 5606 int *peType, 5607 5607 int *piPgPtr, 5608 5608 u8 **paKey, int *pnKey, 5609 5609 u8 **paVal, int *pnVal, 5610 - Blob *pBlob 5610 + LsmBlob *pBlob 5611 5611 ){ 5612 5612 u8 *aData; int nData; /* Page data */ 5613 5613 u8 *aKey; int nKey = 0; /* Key */ 5614 5614 u8 *aVal = 0; int nVal = 0; /* Value */ 5615 5615 int eType; 5616 5616 int iPgPtr; 5617 5617 Page *pRef = 0; /* Pointer to page iRef */ ................................................................................ 5621 5621 5622 5622 aCell = pageGetCell(aData, nData, iCell); 5623 5623 eType = *aCell++; 5624 5624 aCell += lsmVarintGet32(aCell, &iPgPtr); 5625 5625 5626 5626 if( eType==0 ){ 5627 5627 int dummy; 5628 - Pgno iRef; /* Page number of referenced page */ 5628 + LsmPgno iRef; /* Page number of referenced page */ 5629 5629 aCell += lsmVarintGet64(aCell, &iRef); 5630 5630 if( bIndirect ){ 5631 5631 lsmFsDbPageGet(pDb->pFS, pSeg, iRef, &pRef); 5632 5632 pageGetKeyCopy(pDb->pEnv, pSeg, pRef, 0, &dummy, pBlob); 5633 5633 aKey = (u8 *)pBlob->pData; 5634 5634 nKey = pBlob->nData; 5635 5635 lsmFsPageRelease(pRef); ................................................................................ 5667 5667 #define INFO_PAGE_DUMP_DATA 0x01 5668 5668 #define INFO_PAGE_DUMP_VALUES 0x02 5669 5669 #define INFO_PAGE_DUMP_HEX 0x04 5670 5670 #define INFO_PAGE_DUMP_INDIRECT 0x08 5671 5671 5672 5672 static int infoPageDump( 5673 5673 lsm_db *pDb, /* Database handle */ 5674 - Pgno iPg, /* Page number of page to dump */ 5674 + LsmPgno iPg, /* Page number of page to dump */ 5675 5675 int flags, 5676 5676 char **pzOut /* OUT: lsmMalloc'd string */ 5677 5677 ){ 5678 5678 int rc = LSM_OK; /* Return code */ 5679 5679 Page *pPg = 0; /* Handle for page iPg */ 5680 5680 int i, j; /* Loop counters */ 5681 5681 const int perLine = 16; /* Bytes per line in the raw hex dump */ ................................................................................ 5708 5708 ** to pass a NULL in place of the segment pointer as the second argument 5709 5709 ** to lsmFsDbPageGet() here. */ 5710 5710 if( rc==LSM_OK ){ 5711 5711 rc = lsmFsDbPageGet(pDb->pFS, 0, iPg, &pPg); 5712 5712 } 5713 5713 5714 5714 if( rc==LSM_OK ){ 5715 - Blob blob = {0, 0, 0, 0}; 5715 + LsmBlob blob = {0, 0, 0, 0}; 5716 5716 int nKeyWidth = 0; 5717 5717 LsmString str; 5718 5718 int nRec; 5719 5719 int iPtr; 5720 5720 int flags2; 5721 5721 int iCell; 5722 5722 u8 *aData; int nData; /* Page data and size thereof */ ................................................................................ 5743 5743 if( bHex ) nKeyWidth = nKeyWidth * 2; 5744 5744 5745 5745 for(iCell=0; iCell<nRec; iCell++){ 5746 5746 u8 *aKey; int nKey = 0; /* Key */ 5747 5747 u8 *aVal; int nVal = 0; /* Value */ 5748 5748 int iPgPtr; 5749 5749 int eType; 5750 - Pgno iAbsPtr; 5750 + LsmPgno iAbsPtr; 5751 5751 char zFlags[8]; 5752 5752 5753 5753 infoCellDump(pDb, pSeg, bIndirect, pPg, iCell, &eType, &iPgPtr, 5754 5754 &aKey, &nKey, &aVal, &nVal, &blob 5755 5755 ); 5756 5756 iAbsPtr = iPgPtr + ((flags2 & SEGMENT_BTREE_FLAG) ? 0 : iPtr); 5757 5757 ................................................................................ 5809 5809 } 5810 5810 5811 5811 return rc; 5812 5812 } 5813 5813 5814 5814 int lsmInfoPageDump( 5815 5815 lsm_db *pDb, /* Database handle */ 5816 - Pgno iPg, /* Page number of page to dump */ 5816 + LsmPgno iPg, /* Page number of page to dump */ 5817 5817 int bHex, /* True to output key/value in hex form */ 5818 5818 char **pzOut /* OUT: lsmMalloc'd string */ 5819 5819 ){ 5820 5820 int flags = INFO_PAGE_DUMP_DATA | INFO_PAGE_DUMP_VALUES; 5821 5821 if( bHex ) flags |= INFO_PAGE_DUMP_HEX; 5822 5822 return infoPageDump(pDb, iPg, flags, pzOut); 5823 5823 } ................................................................................ 5985 5985 iHdr = SEGMENT_EOF(nOrig, nEntry); 5986 5986 memmove(&aData[iHdr + (nData-nOrig)], &aData[iHdr], nOrig-iHdr); 5987 5987 } 5988 5988 5989 5989 #ifdef LSM_DEBUG_EXPENSIVE 5990 5990 static void assertRunInOrder(lsm_db *pDb, Segment *pSeg){ 5991 5991 Page *pPg = 0; 5992 - Blob blob1 = {0, 0, 0, 0}; 5993 - Blob blob2 = {0, 0, 0, 0}; 5992 + LsmBlob blob1 = {0, 0, 0, 0}; 5993 + LsmBlob blob2 = {0, 0, 0, 0}; 5994 5994 5995 5995 lsmFsDbPageGet(pDb->pFS, pSeg, pSeg->iFirst, &pPg); 5996 5996 while( pPg ){ 5997 5997 u8 *aData; int nData; 5998 5998 Page *pNext; 5999 5999 6000 6000 aData = lsmFsPageData(pPg, &nData); ................................................................................ 6048 6048 Segment *pOne, /* Segment containing pointers */ 6049 6049 Segment *pTwo, /* Segment containing pointer targets */ 6050 6050 int bRhs /* True if pTwo may have been Gobble()d */ 6051 6051 ){ 6052 6052 int rc = LSM_OK; /* Error code */ 6053 6053 SegmentPtr ptr1; /* Iterates through pOne */ 6054 6054 SegmentPtr ptr2; /* Iterates through pTwo */ 6055 - Pgno iPrev; 6055 + LsmPgno iPrev; 6056 6056 6057 6057 assert( pOne && pTwo ); 6058 6058 6059 6059 memset(&ptr1, 0, sizeof(ptr1)); 6060 6060 memset(&ptr2, 0, sizeof(ptr1)); 6061 6061 ptr1.pSeg = pOne; 6062 6062 ptr2.pSeg = pTwo; ................................................................................ 6071 6071 } 6072 6072 6073 6073 if( rc==LSM_OK && ptr1.nCell>0 ){ 6074 6074 rc = segmentPtrLoadCell(&ptr1, 0); 6075 6075 } 6076 6076 6077 6077 while( rc==LSM_OK && ptr2.pPg ){ 6078 - Pgno iThis; 6078 + LsmPgno iThis; 6079 6079 6080 6080 /* Advance to the next page of segment pTwo that contains at least 6081 6081 ** one cell. Break out of the loop if the iterator reaches EOF. */ 6082 6082 do{ 6083 6083 rc = segmentPtrNextPage(&ptr2, 1); 6084 6084 assert( rc==LSM_OK ); 6085 6085 }while( rc==LSM_OK && ptr2.pPg && ptr2.nCell==0 ); ................................................................................ 6133 6133 */ 6134 6134 static int assertBtreeOk( 6135 6135 lsm_db *pDb, 6136 6136 Segment *pSeg 6137 6137 ){ 6138 6138 int rc = LSM_OK; /* Return code */ 6139 6139 if( pSeg->iRoot ){ 6140 - Blob blob = {0, 0, 0}; /* Buffer used to cache overflow keys */ 6140 + LsmBlob blob = {0, 0, 0}; /* Buffer used to cache overflow keys */ 6141 6141 FileSystem *pFS = pDb->pFS; /* File system to read from */ 6142 6142 Page *pPg = 0; /* Main run page */ 6143 6143 BtreeCursor *pCsr = 0; /* Btree cursor */ 6144 6144 6145 6145 rc = btreeCursorNew(pDb, pSeg, &pCsr); 6146 6146 if( rc==LSM_OK ){ 6147 6147 rc = btreeCursorFirst(pCsr);
Added ext/lsm1/tool/mklsm1c.tcl.
1 +#!/bin/sh 2 +# restart with tclsh \ 3 +exec tclsh "$0" "$@" 4 + 5 +set srcdir [file dirname [file dirname [info script]]] 6 +set G(src) [string map [list %dir% $srcdir] { 7 + %dir%/lsm.h 8 + %dir%/lsmInt.h 9 + %dir%/lsm_vtab.c 10 + %dir%/lsm_ckpt.c 11 + %dir%/lsm_file.c 12 + %dir%/lsm_log.c 13 + %dir%/lsm_main.c 14 + %dir%/lsm_mem.c 15 + %dir%/lsm_mutex.c 16 + %dir%/lsm_shared.c 17 + %dir%/lsm_sorted.c 18 + %dir%/lsm_str.c 19 + %dir%/lsm_tree.c 20 + %dir%/lsm_unix.c 21 + %dir%/lsm_varint.c 22 + %dir%/lsm_win32.c 23 +}] 24 + 25 +set G(hdr) { 26 + 27 +#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_LSM1) 28 + 29 +#if !defined(NDEBUG) && !defined(SQLITE_DEBUG) 30 +# define NDEBUG 1 31 +#endif 32 +#if defined(NDEBUG) && defined(SQLITE_DEBUG) 33 +# undef NDEBUG 34 +#endif 35 + 36 +} 37 + 38 +set G(footer) { 39 + 40 +#endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_LSM1) */ 41 +} 42 + 43 +#------------------------------------------------------------------------- 44 +# Read and return the entire contents of text file $zFile from disk. 45 +# 46 +proc readfile {zFile} { 47 + set fd [open $zFile] 48 + set data [read $fd] 49 + close $fd 50 + return $data 51 +} 52 + 53 +proc lsm1c_init {zOut} { 54 + global G 55 + set G(fd) stdout 56 + set G(fd) [open $zOut w] 57 + 58 + puts -nonewline $G(fd) $G(hdr) 59 +} 60 + 61 +proc lsm1c_printfile {zIn} { 62 + global G 63 + set data [readfile $zIn] 64 + set zTail [file tail $zIn] 65 + puts $G(fd) "#line 1 \"$zTail\"" 66 + 67 + foreach line [split $data "\n"] { 68 + if {[regexp {^# *include.*lsm} $line]} { 69 + set line "/* $line */" 70 + } elseif { [regexp {^(const )?[a-zA-Z][a-zA-Z0-9]* [*]?lsm[^_]} $line] } { 71 + set line "static $line" 72 + } 73 + puts $G(fd) $line 74 + } 75 +} 76 + 77 +proc lsm1c_close {} { 78 + global G 79 + puts -nonewline $G(fd) $G(footer) 80 + if {$G(fd)!="stdout"} { 81 + close $G(fd) 82 + } 83 +} 84 + 85 + 86 +lsm1c_init lsm1.c 87 +foreach f $G(src) { lsm1c_printfile $f } 88 +lsm1c_close
Changes to ext/misc/rot13.c.
43 43 sqlite3_context *context, 44 44 int argc, 45 45 sqlite3_value **argv 46 46 ){ 47 47 const unsigned char *zIn; 48 48 int nIn; 49 49 unsigned char *zOut; 50 - char *zToFree = 0; 50 + unsigned char *zToFree = 0; 51 51 int i; 52 - char zTemp[100]; 52 + unsigned char zTemp[100]; 53 53 assert( argc==1 ); 54 54 if( sqlite3_value_type(argv[0])==SQLITE_NULL ) return; 55 55 zIn = (const unsigned char*)sqlite3_value_text(argv[0]); 56 56 nIn = sqlite3_value_bytes(argv[0]); 57 57 if( nIn<sizeof(zTemp)-1 ){ 58 58 zOut = zTemp; 59 59 }else{ 60 - zOut = zToFree = sqlite3_malloc( nIn+1 ); 60 + zOut = zToFree = (unsigned char*)sqlite3_malloc64( nIn+1 ); 61 61 if( zOut==0 ){ 62 62 sqlite3_result_error_nomem(context); 63 63 return; 64 64 } 65 65 } 66 66 for(i=0; i<nIn; i++) zOut[i] = rot13(zIn[i]); 67 67 zOut[i] = 0;
Changes to main.mk.
259 259 fts5parse.c \ 260 260 $(TOP)/ext/fts5/fts5_storage.c \ 261 261 $(TOP)/ext/fts5/fts5_tokenize.c \ 262 262 $(TOP)/ext/fts5/fts5_unicode2.c \ 263 263 $(TOP)/ext/fts5/fts5_varint.c \ 264 264 $(TOP)/ext/fts5/fts5_vocab.c \ 265 265 266 +LSM1_SRC = \ 267 + $(TOP)/ext/lsm1/lsm.h \ 268 + $(TOP)/ext/lsm1/lsmInt.h \ 269 + $(TOP)/ext/lsm1/lsm_ckpt.c \ 270 + $(TOP)/ext/lsm1/lsm_file.c \ 271 + $(TOP)/ext/lsm1/lsm_log.c \ 272 + $(TOP)/ext/lsm1/lsm_main.c \ 273 + $(TOP)/ext/lsm1/lsm_mem.c \ 274 + $(TOP)/ext/lsm1/lsm_mutex.c \ 275 + $(TOP)/ext/lsm1/lsm_shared.c \ 276 + $(TOP)/ext/lsm1/lsm_sorted.c \ 277 + $(TOP)/ext/lsm1/lsm_str.c \ 278 + $(TOP)/ext/lsm1/lsm_tree.c \ 279 + $(TOP)/ext/lsm1/lsm_unix.c \ 280 + $(TOP)/ext/lsm1/lsm_varint.c \ 281 + $(TOP)/ext/lsm1/lsm_vtab.c \ 282 + $(TOP)/ext/lsm1/lsm_win32.c 283 + 266 284 267 285 # Generated source code files 268 286 # 269 287 SRC += \ 270 288 keywordhash.h \ 271 289 opcodes.c \ 272 290 opcodes.h \ ................................................................................ 762 780 763 781 fts5parse.h: fts5parse.c 764 782 765 783 fts5.c: $(FTS5_SRC) $(FTS5_HDR) 766 784 tclsh $(TOP)/ext/fts5/tool/mkfts5c.tcl 767 785 cp $(TOP)/ext/fts5/fts5.h . 768 786 787 +lsm1.c: $(LSM1_SRC) 788 + tclsh $(TOP)/ext/lsm1/tool/mklsm1c.tcl 789 + cp $(TOP)/ext/lsm1/lsm.h . 790 + 769 791 userauth.o: $(TOP)/ext/userauth/userauth.c $(HDR) $(EXTHDR) 770 792 $(TCCX) -DSQLITE_CORE -c $(TOP)/ext/userauth/userauth.c 771 793 772 794 sqlite3session.o: $(TOP)/ext/session/sqlite3session.c $(HDR) $(EXTHDR) 773 795 $(TCCX) -DSQLITE_CORE -c $(TOP)/ext/session/sqlite3session.c 774 796 775 797 sqlite3rbu.o: $(TOP)/ext/rbu/sqlite3rbu.c $(HDR) $(EXTHDR) ................................................................................ 1015 1037 rm -f sqlite3_analyzer sqlite3_analyzer.exe sqlite3_analyzer.c 1016 1038 rm -f sqlite-*-output.vsix 1017 1039 rm -f mptester mptester.exe 1018 1040 rm -f fuzzershell fuzzershell.exe 1019 1041 rm -f fuzzcheck fuzzcheck.exe 1020 1042 rm -f sqldiff sqldiff.exe 1021 1043 rm -f fts5.* fts5parse.* 1044 + rm -f lsm.h lsm1.c
Changes to src/build.c.
4360 4360 for(i=0; i<nCol; i++){ 4361 4361 const char *zColl = pIdx->azColl[i]; 4362 4362 pKey->aColl[i] = zColl==sqlite3StrBINARY ? 0 : 4363 4363 sqlite3LocateCollSeq(pParse, zColl); 4364 4364 pKey->aSortOrder[i] = pIdx->aSortOrder[i]; 4365 4365 } 4366 4366 if( pParse->nErr ){ 4367 + assert( pParse->rc==SQLITE_ERROR_MISSING_COLLSEQ ); 4368 + if( pIdx->bNoQuery==0 ){ 4369 + /* Deactivate the index because it contains an unknown collating 4370 + ** sequence. The only way to reactive the index is to reload the 4371 + ** schema. Adding the missing collating sequence later does not 4372 + ** reactive the index. The application had the chance to register 4373 + ** the missing index using the collation-needed callback. For 4374 + ** simplicity, SQLite will not give the application a second chance. 4375 + */ 4376 + pIdx->bNoQuery = 1; 4377 + pParse->rc = SQLITE_ERROR_RETRY; 4378 + } 4367 4379 sqlite3KeyInfoUnref(pKey); 4368 4380 pKey = 0; 4369 4381 } 4370 4382 } 4371 4383 return pKey; 4372 4384 } 4373 4385
Changes to src/callback.c.
101 101 } 102 102 if( p && !p->xCmp && synthCollSeq(db, p) ){ 103 103 p = 0; 104 104 } 105 105 assert( !p || p->xCmp ); 106 106 if( p==0 ){ 107 107 sqlite3ErrorMsg(pParse, "no such collation sequence: %s", zName); 108 + pParse->rc = SQLITE_ERROR_MISSING_COLLSEQ; 108 109 } 109 110 return p; 110 111 } 111 112 112 113 /* 113 114 ** This routine is called on a collation sequence before it is used to 114 115 ** check that it is defined. An undefined collation sequence exists when
Changes to src/date.c.
35 35 ** dates afterwards, depending on locale. Beware of this difference. 36 36 ** 37 37 ** The conversion algorithms are implemented based on descriptions 38 38 ** in the following text: 39 39 ** 40 40 ** Jean Meeus 41 41 ** Astronomical Algorithms, 2nd Edition, 1998 42 -** ISBM 0-943396-61-1 42 +** ISBN 0-943396-61-1 43 43 ** Willmann-Bell, Inc 44 44 ** Richmond, Virginia (USA) 45 45 */ 46 46 #include "sqliteInt.h" 47 47 #include <stdlib.h> 48 48 #include <assert.h> 49 49 #include <time.h>
Changes to src/delete.c.
279 279 280 280 /* Figure out if we have any triggers and if the table being 281 281 ** deleted from is a view 282 282 */ 283 283 #ifndef SQLITE_OMIT_TRIGGER 284 284 pTrigger = sqlite3TriggersExist(pParse, pTab, TK_DELETE, 0, 0); 285 285 isView = pTab->pSelect!=0; 286 - bComplex = pTrigger || sqlite3FkRequired(pParse, pTab, 0, 0); 287 286 #else 288 287 # define pTrigger 0 289 288 # define isView 0 290 289 #endif 290 + bComplex = pTrigger || sqlite3FkRequired(pParse, pTab, 0, 0); 291 291 #ifdef SQLITE_OMIT_VIEW 292 292 # undef isView 293 293 # define isView 0 294 294 #endif 295 295 296 296 #ifdef SQLITE_ENABLE_UPDATE_DELETE_LIMIT 297 297 if( !isView ){
Changes to src/main.c.
18 18 19 19 #ifdef SQLITE_ENABLE_FTS3 20 20 # include "fts3.h" 21 21 #endif 22 22 #ifdef SQLITE_ENABLE_RTREE 23 23 # include "rtree.h" 24 24 #endif 25 -#ifdef SQLITE_ENABLE_ICU 25 +#if defined(SQLITE_ENABLE_ICU) || defined(SQLITE_ENABLE_ICU_COLLATIONS) 26 26 # include "sqliteicu.h" 27 27 #endif 28 28 #ifdef SQLITE_ENABLE_JSON1 29 29 int sqlite3Json1Init(sqlite3*); 30 30 #endif 31 31 #ifdef SQLITE_ENABLE_STMTVTAB 32 32 int sqlite3StmtVtabInit(sqlite3*); ................................................................................ 3046 3046 3047 3047 #ifdef SQLITE_ENABLE_FTS3 /* automatically defined by SQLITE_ENABLE_FTS4 */ 3048 3048 if( !db->mallocFailed && rc==SQLITE_OK ){ 3049 3049 rc = sqlite3Fts3Init(db); 3050 3050 } 3051 3051 #endif 3052 3052 3053 -#ifdef SQLITE_ENABLE_ICU 3053 +#if defined(SQLITE_ENABLE_ICU) || defined(SQLITE_ENABLE_ICU_COLLATIONS) 3054 3054 if( !db->mallocFailed && rc==SQLITE_OK ){ 3055 3055 rc = sqlite3IcuInit(db); 3056 3056 } 3057 3057 #endif 3058 3058 3059 3059 #ifdef SQLITE_ENABLE_RTREE 3060 3060 if( !db->mallocFailed && rc==SQLITE_OK){
Changes to src/os_unix.c.
479 479 #define osMmap ((void*(*)(void*,size_t,int,int,int,off_t))aSyscall[22].pCurrent) 480 480 481 481 #if !defined(SQLITE_OMIT_WAL) || SQLITE_MAX_MMAP_SIZE>0 482 482 { "munmap", (sqlite3_syscall_ptr)munmap, 0 }, 483 483 #else 484 484 { "munmap", (sqlite3_syscall_ptr)0, 0 }, 485 485 #endif 486 -#define osMunmap ((void*(*)(void*,size_t))aSyscall[23].pCurrent) 486 +#define osMunmap ((int(*)(void*,size_t))aSyscall[23].pCurrent) 487 487 488 488 #if HAVE_MREMAP && (!defined(SQLITE_OMIT_WAL) || SQLITE_MAX_MMAP_SIZE>0) 489 489 { "mremap", (sqlite3_syscall_ptr)mremap, 0 }, 490 490 #else 491 491 { "mremap", (sqlite3_syscall_ptr)0, 0 }, 492 492 #endif 493 493 #define osMremap ((void*(*)(void*,size_t,size_t,int,...))aSyscall[24].pCurrent) ................................................................................ 4161 4161 ){ 4162 4162 unixShmNode *pShmNode; /* Apply locks to this open shared-memory segment */ 4163 4163 struct flock f; /* The posix advisory locking structure */ 4164 4164 int rc = SQLITE_OK; /* Result code form fcntl() */ 4165 4165 4166 4166 /* Access to the unixShmNode object is serialized by the caller */ 4167 4167 pShmNode = pFile->pInode->pShmNode; 4168 - assert( sqlite3_mutex_held(pShmNode->mutex) || pShmNode->nRef==0 ); 4168 + assert( pShmNode->nRef==0 || sqlite3_mutex_held(pShmNode->mutex) ); 4169 4169 4170 4170 /* Shared locks never span more than one byte */ 4171 4171 assert( n==1 || lockType!=F_RDLCK ); 4172 4172 4173 4173 /* Locks are within range */ 4174 4174 assert( n>=1 && n<=SQLITE_SHM_NLOCK ); 4175 4175 ................................................................................ 5795 5795 struct statfs fsInfo; 5796 5796 #endif 5797 5797 5798 5798 /* If creating a master or main-file journal, this function will open 5799 5799 ** a file-descriptor on the directory too. The first time unixSync() 5800 5800 ** is called the directory file descriptor will be fsync()ed and close()d. 5801 5801 */ 5802 - int syncDir = (isCreate && ( 5802 + int isNewJrnl = (isCreate && ( 5803 5803 eType==SQLITE_OPEN_MASTER_JOURNAL 5804 5804 || eType==SQLITE_OPEN_MAIN_JOURNAL 5805 5805 || eType==SQLITE_OPEN_WAL 5806 5806 )); 5807 5807 5808 5808 /* If argument zPath is a NULL pointer, this function is required to open 5809 5809 ** a temporary file. Use this buffer to store the file name in. ................................................................................ 5865 5865 /* Database filenames are double-zero terminated if they are not 5866 5866 ** URIs with parameters. Hence, they can always be passed into 5867 5867 ** sqlite3_uri_parameter(). */ 5868 5868 assert( (flags & SQLITE_OPEN_URI) || zName[strlen(zName)+1]==0 ); 5869 5869 5870 5870 }else if( !zName ){ 5871 5871 /* If zName is NULL, the upper layer is requesting a temp file. */ 5872 - assert(isDelete && !syncDir); 5872 + assert(isDelete && !isNewJrnl); 5873 5873 rc = unixGetTempname(pVfs->mxPathname, zTmpname); 5874 5874 if( rc!=SQLITE_OK ){ 5875 5875 return rc; 5876 5876 } 5877 5877 zName = zTmpname; 5878 5878 5879 5879 /* Generated temporary filenames are always double-zero terminated ................................................................................ 5911 5911 flags |= SQLITE_OPEN_READONLY; 5912 5912 openFlags |= O_RDONLY; 5913 5913 isReadonly = 1; 5914 5914 fd = robust_open(zName, openFlags, openMode); 5915 5915 } 5916 5916 if( fd<0 ){ 5917 5917 rc = unixLogError(SQLITE_CANTOPEN_BKPT, "open", zName); 5918 + /* If unable to create a journal, change the error code to 5919 + ** indicate that the directory permissions are wrong. */ 5920 + if( isNewJrnl && osAccess(zName, F_OK) ) rc = SQLITE_READONLY_DIRECTORY; 5918 5921 goto open_finished; 5919 5922 } 5920 5923 5921 5924 /* If this process is running as root and if creating a new rollback 5922 5925 ** journal or WAL file, set the ownership of the journal or WAL to be 5923 5926 ** the same as the original database. 5924 5927 */ ................................................................................ 5970 5973 #endif 5971 5974 5972 5975 /* Set up appropriate ctrlFlags */ 5973 5976 if( isDelete ) ctrlFlags |= UNIXFILE_DELETE; 5974 5977 if( isReadonly ) ctrlFlags |= UNIXFILE_RDONLY; 5975 5978 noLock = eType!=SQLITE_OPEN_MAIN_DB; 5976 5979 if( noLock ) ctrlFlags |= UNIXFILE_NOLOCK; 5977 - if( syncDir ) ctrlFlags |= UNIXFILE_DIRSYNC; 5980 + if( isNewJrnl ) ctrlFlags |= UNIXFILE_DIRSYNC; 5978 5981 if( flags & SQLITE_OPEN_URI ) ctrlFlags |= UNIXFILE_URI; 5979 5982 5980 5983 #if SQLITE_ENABLE_LOCKING_STYLE 5981 5984 #if SQLITE_PREFER_PROXY_LOCKING 5982 5985 isAutoProxy = 1; 5983 5986 #endif 5984 5987 if( isAutoProxy && (zPath!=NULL) && (!noLock) && pVfs->xOpen ){
Changes to src/os_win.c.
3738 3738 int lockType, /* WINSHM_UNLCK, WINSHM_RDLCK, or WINSHM_WRLCK */ 3739 3739 int ofst, /* Offset to first byte to be locked/unlocked */ 3740 3740 int nByte /* Number of bytes to lock or unlock */ 3741 3741 ){ 3742 3742 int rc = 0; /* Result code form Lock/UnlockFileEx() */ 3743 3743 3744 3744 /* Access to the winShmNode object is serialized by the caller */ 3745 - assert( sqlite3_mutex_held(pFile->mutex) || pFile->nRef==0 ); 3745 + assert( pFile->nRef==0 || sqlite3_mutex_held(pFile->mutex) ); 3746 3746 3747 3747 OSTRACE(("SHM-LOCK file=%p, lock=%d, offset=%d, size=%d\n", 3748 3748 pFile->hFile.h, lockType, ofst, nByte)); 3749 3749 3750 3750 /* Release/Acquire the system-level lock */ 3751 3751 if( lockType==WINSHM_UNLCK ){ 3752 3752 rc = winUnlockFile(&pFile->hFile.h, ofst, 0, nByte, 0);
Changes to src/prepare.c.
651 651 sParse.pTriggerPrg = pT->pNext; 652 652 sqlite3DbFree(db, pT); 653 653 } 654 654 655 655 end_prepare: 656 656 657 657 sqlite3ParserReset(&sParse); 658 - rc = sqlite3ApiExit(db, rc); 659 - assert( (rc&db->errMask)==rc ); 660 658 return rc; 661 659 } 662 660 static int sqlite3LockAndPrepare( 663 661 sqlite3 *db, /* Database handle. */ 664 662 const char *zSql, /* UTF-8 encoded SQL statement. */ 665 663 int nBytes, /* Length of zSql in bytes. */ 666 664 u32 prepFlags, /* Zero or more SQLITE_PREPARE_* flags */ 667 665 Vdbe *pOld, /* VM being reprepared */ 668 666 sqlite3_stmt **ppStmt, /* OUT: A pointer to the prepared statement */ 669 667 const char **pzTail /* OUT: End of parsed string */ 670 668 ){ 671 669 int rc; 670 + int cnt = 0; 672 671 673 672 #ifdef SQLITE_ENABLE_API_ARMOR 674 673 if( ppStmt==0 ) return SQLITE_MISUSE_BKPT; 675 674 #endif 676 675 *ppStmt = 0; 677 676 if( !sqlite3SafetyCheckOk(db)||zSql==0 ){ 678 677 return SQLITE_MISUSE_BKPT; 679 678 } 680 679 sqlite3_mutex_enter(db->mutex); 681 680 sqlite3BtreeEnterAll(db); 682 - rc = sqlite3Prepare(db, zSql, nBytes, prepFlags, pOld, ppStmt, pzTail); 683 - if( rc==SQLITE_SCHEMA ){ 684 - sqlite3ResetOneSchema(db, -1); 685 - sqlite3_finalize(*ppStmt); 681 + do{ 682 + /* Make multiple attempts to compile the SQL, until it either succeeds 683 + ** or encounters a permanent error. A schema problem after one schema 684 + ** reset is considered a permanent error. */ 686 685 rc = sqlite3Prepare(db, zSql, nBytes, prepFlags, pOld, ppStmt, pzTail); 687 - } 686 + assert( rc==SQLITE_OK || *ppStmt==0 ); 687 + }while( rc==SQLITE_ERROR_RETRY 688 + || (rc==SQLITE_SCHEMA && (sqlite3ResetOneSchema(db,-1), cnt++)==0) ); 688 689 sqlite3BtreeLeaveAll(db); 690 + rc = sqlite3ApiExit(db, rc); 691 + assert( (rc&db->errMask)==rc ); 689 692 sqlite3_mutex_leave(db->mutex); 690 - assert( rc==SQLITE_OK || *ppStmt==0 ); 691 693 return rc; 692 694 } 693 695 694 696 /* 695 697 ** Rerun the compilation of a statement after a schema change. 696 698 ** 697 699 ** If the statement is successfully recompiled, return SQLITE_OK. Otherwise,
Changes to src/sqlite.h.in.
466 466 ** support for additional result codes that provide more detailed information 467 467 ** about errors. These [extended result codes] are enabled or disabled 468 468 ** on a per database connection basis using the 469 469 ** [sqlite3_extended_result_codes()] API. Or, the extended code for 470 470 ** the most recent error can be obtained using 471 471 ** [sqlite3_extended_errcode()]. 472 472 */ 473 +#define SQLITE_ERROR_MISSING_COLLSEQ (SQLITE_ERROR | (1<<8)) 474 +#define SQLITE_ERROR_RETRY (SQLITE_ERROR | (2<<8)) 473 475 #define SQLITE_IOERR_READ (SQLITE_IOERR | (1<<8)) 474 476 #define SQLITE_IOERR_SHORT_READ (SQLITE_IOERR | (2<<8)) 475 477 #define SQLITE_IOERR_WRITE (SQLITE_IOERR | (3<<8)) 476 478 #define SQLITE_IOERR_FSYNC (SQLITE_IOERR | (4<<8)) 477 479 #define SQLITE_IOERR_DIR_FSYNC (SQLITE_IOERR | (5<<8)) 478 480 #define SQLITE_IOERR_TRUNCATE (SQLITE_IOERR | (6<<8)) 479 481 #define SQLITE_IOERR_FSTAT (SQLITE_IOERR | (7<<8)) ................................................................................ 511 513 #define SQLITE_CANTOPEN_DIRTYWAL (SQLITE_CANTOPEN | (5<<8)) 512 514 #define SQLITE_CORRUPT_VTAB (SQLITE_CORRUPT | (1<<8)) 513 515 #define SQLITE_READONLY_RECOVERY (SQLITE_READONLY | (1<<8)) 514 516 #define SQLITE_READONLY_CANTLOCK (SQLITE_READONLY | (2<<8)) 515 517 #define SQLITE_READONLY_ROLLBACK (SQLITE_READONLY | (3<<8)) 516 518 #define SQLITE_READONLY_DBMOVED (SQLITE_READONLY | (4<<8)) 517 519 #define SQLITE_READONLY_CANTINIT (SQLITE_READONLY | (5<<8)) 520 +#define SQLITE_READONLY_DIRECTORY (SQLITE_READONLY | (6<<8)) 518 521 #define SQLITE_ABORT_ROLLBACK (SQLITE_ABORT | (2<<8)) 519 522 #define SQLITE_CONSTRAINT_CHECK (SQLITE_CONSTRAINT | (1<<8)) 520 523 #define SQLITE_CONSTRAINT_COMMITHOOK (SQLITE_CONSTRAINT | (2<<8)) 521 524 #define SQLITE_CONSTRAINT_FOREIGNKEY (SQLITE_CONSTRAINT | (3<<8)) 522 525 #define SQLITE_CONSTRAINT_FUNCTION (SQLITE_CONSTRAINT | (4<<8)) 523 526 #define SQLITE_CONSTRAINT_NOTNULL (SQLITE_CONSTRAINT | (5<<8)) 524 527 #define SQLITE_CONSTRAINT_PRIMARYKEY (SQLITE_CONSTRAINT | (6<<8))
Changes to src/sqliteInt.h.
2168 2168 unsigned idxType:2; /* 1==UNIQUE, 2==PRIMARY KEY, 0==CREATE INDEX */ 2169 2169 unsigned bUnordered:1; /* Use this index for == or IN queries only */ 2170 2170 unsigned uniqNotNull:1; /* True if UNIQUE and NOT NULL for all columns */ 2171 2171 unsigned isResized:1; /* True if resizeIndexObject() has been called */ 2172 2172 unsigned isCovering:1; /* True if this is a covering index */ 2173 2173 unsigned noSkipScan:1; /* Do not try to use skip-scan if true */ 2174 2174 unsigned hasStat1:1; /* aiRowLogEst values come from sqlite_stat1 */ 2175 + unsigned bNoQuery:1; /* Do not use this index to optimize queries */ 2175 2176 #ifdef SQLITE_ENABLE_STAT3_OR_STAT4 2176 2177 int nSample; /* Number of elements in aSample[] */ 2177 2178 int nSampleCol; /* Size of IndexSample.anEq[] and so on */ 2178 2179 tRowcnt *aAvgEq; /* Average nEq values for keys not in aSample */ 2179 2180 IndexSample *aSample; /* Samples of the left-most key */ 2180 2181 tRowcnt *aiRowEst; /* Non-logarithmic stat1 data for this index */ 2181 2182 tRowcnt nRowEst0; /* Non-logarithmic number of rows in the index */ ................................................................................ 2980 2981 int nRangeReg; /* Size of the temporary register block */ 2981 2982 int iRangeReg; /* First register in temporary register block */ 2982 2983 int nErr; /* Number of errors seen */ 2983 2984 int nTab; /* Number of previously allocated VDBE cursors */ 2984 2985 int nMem; /* Number of memory cells used so far */ 2985 2986 int nOpAlloc; /* Number of slots allocated for Vdbe.aOp[] */ 2986 2987 int szOpAlloc; /* Bytes of memory space allocated for Vdbe.aOp[] */ 2987 - int iSelfTab; /* Table for associated with an index on expr, or negative 2988 + int iSelfTab; /* Table associated with an index on expr, or negative 2988 2989 ** of the base register during check-constraint eval */ 2989 2990 int iCacheLevel; /* ColCache valid when aColCache[].iLevel<=iCacheLevel */ 2990 2991 int iCacheCnt; /* Counter used to generate aColCache[].lru values */ 2991 2992 int nLabel; /* Number of labels used */ 2992 2993 int *aLabel; /* Space to hold the labels */ 2993 2994 ExprList *pConstExpr;/* Constant expressions */ 2994 2995 Token constraintName;/* Name of the constraint currently being parsed */
Changes to src/test_config.c.
424 424 #endif 425 425 426 426 #ifdef SQLITE_ENABLE_ICU 427 427 Tcl_SetVar2(interp, "sqlite_options", "icu", "1", TCL_GLOBAL_ONLY); 428 428 #else 429 429 Tcl_SetVar2(interp, "sqlite_options", "icu", "0", TCL_GLOBAL_ONLY); 430 430 #endif 431 + 432 +#ifdef SQLITE_ENABLE_ICU_COLLATIONS 433 + Tcl_SetVar2(interp, "sqlite_options", "icu_collations", "1", TCL_GLOBAL_ONLY); 434 +#else 435 + Tcl_SetVar2(interp, "sqlite_options", "icu_collations", "0", TCL_GLOBAL_ONLY); 436 +#endif 431 437 432 438 #ifdef SQLITE_OMIT_INCRBLOB 433 439 Tcl_SetVar2(interp, "sqlite_options", "incrblob", "0", TCL_GLOBAL_ONLY); 434 440 #else 435 441 Tcl_SetVar2(interp, "sqlite_options", "incrblob", "1", TCL_GLOBAL_ONLY); 436 442 #endif /* SQLITE_OMIT_AUTOVACUUM */ 437 443
Changes to src/vdbemem.c.
1317 1317 sqlite3_value *pVal = 0; 1318 1318 int negInt = 1; 1319 1319 const char *zNeg = ""; 1320 1320 int rc = SQLITE_OK; 1321 1321 1322 1322 assert( pExpr!=0 ); 1323 1323 while( (op = pExpr->op)==TK_UPLUS || op==TK_SPAN ) pExpr = pExpr->pLeft; 1324 +#if defined(SQLITE_ENABLE_STAT3_OR_STAT4) 1324 1325 if( op==TK_REGISTER ) op = pExpr->op2; 1326 +#else 1327 + if( NEVER(op==TK_REGISTER) ) op = pExpr->op2; 1328 +#endif 1325 1329 1326 1330 /* Compressed expressions only appear when parsing the DEFAULT clause 1327 1331 ** on a table column definition, and hence only when pCtx==0. This 1328 1332 ** check ensures that an EP_TokenOnly expression is never passed down 1329 1333 ** into valueFromFunction(). */ 1330 1334 assert( (pExpr->flags & EP_TokenOnly)==0 || pCtx==0 ); 1331 1335 ................................................................................ 1412 1416 } 1413 1417 #endif 1414 1418 1415 1419 *ppVal = pVal; 1416 1420 return rc; 1417 1421 1418 1422 no_mem: 1419 - sqlite3OomFault(db); 1423 +#ifdef SQLITE_ENABLE_STAT3_OR_STAT4 1424 + if( pCtx==0 || pCtx->pParse->nErr==0 ) 1425 +#endif 1426 + sqlite3OomFault(db); 1420 1427 sqlite3DbFree(db, zVal); 1421 1428 assert( *ppVal==0 ); 1422 1429 #ifdef SQLITE_ENABLE_STAT3_OR_STAT4 1423 1430 if( pCtx==0 ) sqlite3ValueFree(pVal); 1424 1431 #else 1425 1432 assert( pCtx==0 ); sqlite3ValueFree(pVal); 1426 1433 #endif
Changes to src/where.c.
2875 2875 pProbe=(pSrc->pIBIndex ? 0 : pProbe->pNext), iSortIdx++ 2876 2876 ){ 2877 2877 if( pProbe->pPartIdxWhere!=0 2878 2878 && !whereUsablePartialIndex(pSrc->iCursor, pWC, pProbe->pPartIdxWhere) ){ 2879 2879 testcase( pNew->iTab!=pSrc->iCursor ); /* See ticket [98d973b8f5] */ 2880 2880 continue; /* Partial index inappropriate for this query */ 2881 2881 } 2882 + if( pProbe->bNoQuery ) continue; 2882 2883 rSize = pProbe->aiRowLogEst[0]; 2883 2884 pNew->u.btree.nEq = 0; 2884 2885 pNew->u.btree.nBtm = 0; 2885 2886 pNew->u.btree.nTop = 0; 2886 2887 pNew->nSkip = 0; 2887 2888 pNew->nLTerm = 0; 2888 2889 pNew->iSortIdx = 0;
Changes to src/wherecode.c.
1686 1686 if( sqlite3ExprIsVector(pRight)==0 ){ 1687 1687 disableTerm(pLevel, pRangeEnd); 1688 1688 }else{ 1689 1689 endEq = 1; 1690 1690 } 1691 1691 }else if( bStopAtNull ){ 1692 1692 sqlite3VdbeAddOp2(v, OP_Null, 0, regBase+nEq); 1693 + sqlite3ExprCacheRemove(pParse, regBase+nEq, 1); 1693 1694 endEq = 0; 1694 1695 nConstraint++; 1695 1696 } 1696 1697 sqlite3DbFree(db, zStartAff); 1697 1698 sqlite3DbFree(db, zEndAff); 1698 1699 1699 1700 /* Top of the loop body */
Changes to test/icu.test.
11 11 # 12 12 # $Id: icu.test,v 1.2 2008/07/12 14:52:20 drh Exp $ 13 13 # 14 14 15 15 set testdir [file dirname $argv0] 16 16 source $testdir/tester.tcl 17 17 18 -ifcapable !icu { 18 +ifcapable !icu&&!icu_collations { 19 19 finish_test 20 20 return 21 21 } 22 22 23 23 # Create a table to work with. 24 24 # 25 25 execsql {CREATE TABLE test1(i1 int, i2 int, r1 real, r2 real, t1 text, t2 text)} ................................................................................ 31 31 UPDATE test1 SET %s; 32 32 SELECT %s FROM test1; 33 33 ROLLBACK; 34 34 }] 0 35 35 } $settings $expr] $result 36 36 } 37 37 38 -# Tests of the REGEXP operator. 39 -# 40 -test_expr icu-1.1 {i1='hello'} {i1 REGEXP 'hello'} 1 41 -test_expr icu-1.2 {i1='hello'} {i1 REGEXP '.ello'} 1 42 -test_expr icu-1.3 {i1='hello'} {i1 REGEXP '.ell'} 0 43 -test_expr icu-1.4 {i1='hello'} {i1 REGEXP '.ell.*'} 1 44 -test_expr icu-1.5 {i1=NULL} {i1 REGEXP '.ell.*'} {} 38 +ifcapable icu { 39 + 40 + # Tests of the REGEXP operator. 41 + # 42 + test_expr icu-1.1 {i1='hello'} {i1 REGEXP 'hello'} 1 43 + test_expr icu-1.2 {i1='hello'} {i1 REGEXP '.ello'} 1 44 + test_expr icu-1.3 {i1='hello'} {i1 REGEXP '.ell'} 0 45 + test_expr icu-1.4 {i1='hello'} {i1 REGEXP '.ell.*'} 1 46 + test_expr icu-1.5 {i1=NULL} {i1 REGEXP '.ell.*'} {} 47 + 48 + # Some non-ascii characters with defined case mappings 49 + # 50 + set ::EGRAVE "\xC8" 51 + set ::egrave "\xE8" 52 + 53 + set ::OGRAVE "\xD2" 54 + set ::ograve "\xF2" 55 + 56 + # That German letter that looks a bit like a B. The 57 + # upper-case version of which is "SS" (two characters). 58 + # 59 + set ::szlig "\xDF" 45 60 46 -# Some non-ascii characters with defined case mappings 47 -# 48 -set ::EGRAVE "\xC8" 49 -set ::egrave "\xE8" 61 + # Tests of the upper()/lower() functions. 62 + # 63 + test_expr icu-2.1 {i1='HellO WorlD'} {upper(i1)} {HELLO WORLD} 64 + test_expr icu-2.2 {i1='HellO WorlD'} {lower(i1)} {hello world} 65 + test_expr icu-2.3 {i1=$::egrave} {lower(i1)} $::egrave 66 + test_expr icu-2.4 {i1=$::egrave} {upper(i1)} $::EGRAVE 67 + test_expr icu-2.5 {i1=$::ograve} {lower(i1)} $::ograve 68 + test_expr icu-2.6 {i1=$::ograve} {upper(i1)} $::OGRAVE 69 + test_expr icu-2.3 {i1=$::EGRAVE} {lower(i1)} $::egrave 70 + test_expr icu-2.4 {i1=$::EGRAVE} {upper(i1)} $::EGRAVE 71 + test_expr icu-2.5 {i1=$::OGRAVE} {lower(i1)} $::ograve 72 + test_expr icu-2.6 {i1=$::OGRAVE} {upper(i1)} $::OGRAVE 50 73 51 -set ::OGRAVE "\xD2" 52 -set ::ograve "\xF2" 53 - 54 -# That German letter that looks a bit like a B. The 55 -# upper-case version of which is "SS" (two characters). 56 -# 57 -set ::szlig "\xDF" 74 + test_expr icu-2.7 {i1=$::szlig} {upper(i1)} "SS" 75 + test_expr icu-2.8 {i1='SS'} {lower(i1)} "ss" 58 76 59 -# Tests of the upper()/lower() functions. 60 -# 61 -test_expr icu-2.1 {i1='HellO WorlD'} {upper(i1)} {HELLO WORLD} 62 -test_expr icu-2.2 {i1='HellO WorlD'} {lower(i1)} {hello world} 63 -test_expr icu-2.3 {i1=$::egrave} {lower(i1)} $::egrave 64 -test_expr icu-2.4 {i1=$::egrave} {upper(i1)} $::EGRAVE 65 -test_expr icu-2.5 {i1=$::ograve} {lower(i1)} $::ograve 66 -test_expr icu-2.6 {i1=$::ograve} {upper(i1)} $::OGRAVE 67 -test_expr icu-2.3 {i1=$::EGRAVE} {lower(i1)} $::egrave 68 -test_expr icu-2.4 {i1=$::EGRAVE} {upper(i1)} $::EGRAVE 69 -test_expr icu-2.5 {i1=$::OGRAVE} {lower(i1)} $::ograve 70 -test_expr icu-2.6 {i1=$::OGRAVE} {upper(i1)} $::OGRAVE 77 + do_execsql_test icu-2.9 { 78 + SELECT upper(char(0xfb04,0xfb04,0xfb04,0xfb04)); 79 + } {FFLFFLFFLFFL} 71 80 72 -test_expr icu-2.7 {i1=$::szlig} {upper(i1)} "SS" 73 -test_expr icu-2.8 {i1='SS'} {lower(i1)} "ss" 74 - 75 -do_execsql_test icu-2.9 { 76 - SELECT upper(char(0xfb04,0xfb04,0xfb04,0xfb04)); 77 -} {FFLFFLFFLFFL} 78 - 79 -# In turkish (locale="tr_TR"), the lower case version of I 80 -# is "small dotless i" (code point 0x131 (decimal 305)). 81 -# 82 -set ::small_dotless_i "\u0131" 83 -test_expr icu-3.1 {i1='I'} {lower(i1)} "i" 84 -test_expr icu-3.2 {i1='I'} {lower(i1, 'tr_tr')} $::small_dotless_i 85 -test_expr icu-3.3 {i1='I'} {lower(i1, 'en_AU')} "i" 81 + # In turkish (locale="tr_TR"), the lower case version of I 82 + # is "small dotless i" (code point 0x131 (decimal 305)). 83 + # 84 + set ::small_dotless_i "\u0131" 85 + test_expr icu-3.1 {i1='I'} {lower(i1)} "i" 86 + test_expr icu-3.2 {i1='I'} {lower(i1, 'tr_tr')} $::small_dotless_i 87 + test_expr icu-3.3 {i1='I'} {lower(i1, 'en_AU')} "i" 88 +} 86 89 87 90 #-------------------------------------------------------------------- 88 91 # Test the collation sequence function. 89 92 # 90 93 do_test icu-4.1 { 91 94 execsql { 92 95 CREATE TABLE fruit(name); ................................................................................ 120 123 121 124 #------------------------------------------------------------------------- 122 125 # Test that it is not possible to call the ICU regex() function with 123 126 # anything other than exactly two arguments. See also: 124 127 # 125 128 # http://src.chromium.org/viewvc/chrome/trunk/src/third_party/sqlite/icu-regexp.patch?revision=34807&view=markup 126 129 # 127 -do_catchsql_test icu-5.1 { SELECT regexp('a[abc]c.*', 'abc') } {0 1} 128 -do_catchsql_test icu-5.2 { 129 - SELECT regexp('a[abc]c.*') 130 -} {1 {wrong number of arguments to function regexp()}} 131 -do_catchsql_test icu-5.3 { 132 - SELECT regexp('a[abc]c.*', 'abc', 'c') 133 -} {1 {wrong number of arguments to function regexp()}} 134 -do_catchsql_test icu-5.4 { 135 - SELECT 'abc' REGEXP 'a[abc]c.*' 136 -} {0 1} 137 -do_catchsql_test icu-5.4 { SELECT 'abc' REGEXP } {1 {near " ": syntax error}} 138 -do_catchsql_test icu-5.5 { SELECT 'abc' REGEXP, 1 } {1 {near ",": syntax error}} 139 - 140 - 141 -do_malloc_test icu-6.10 -sqlbody { 142 - SELECT upper(char(0xfb04,0xdf,0xfb04,0xe8,0xfb04)); 130 +ifcapable icu { 131 + do_catchsql_test icu-5.1 { SELECT regexp('a[abc]c.*', 'abc') } {0 1} 132 + do_catchsql_test icu-5.2 { 133 + SELECT regexp('a[abc]c.*') 134 + } {1 {wrong number of arguments to function regexp()}} 135 + do_catchsql_test icu-5.3 { 136 + SELECT regexp('a[abc]c.*', 'abc', 'c') 137 + } {1 {wrong number of arguments to function regexp()}} 138 + do_catchsql_test icu-5.4 { 139 + SELECT 'abc' REGEXP 'a[abc]c.*' 140 + } {0 1} 141 + do_catchsql_test icu-5.4 {SELECT 'abc' REGEXP } {1 {near " ": syntax error}} 142 + do_catchsql_test icu-5.5 {SELECT 'abc' REGEXP, 1} {1 {near ",": syntax error}} 143 + 144 + do_malloc_test icu-6.10 -sqlbody { 145 + SELECT upper(char(0xfb04,0xdf,0xfb04,0xe8,0xfb04)); 146 + } 143 147 } 144 148 145 149 finish_test
Changes to test/limit2.test.
145 145 INSERT INTO t502 VALUES(1, 5); 146 146 INSERT INTO t502 VALUES(2, 4); 147 147 INSERT INTO t502 VALUES(3, 3); 148 148 INSERT INTO t502 VALUES(4, 6); 149 149 INSERT INTO t502 VALUES(5, 1); 150 150 SELECT j FROM t502 WHERE i IN (1,2,3,4,5) ORDER BY j LIMIT 3; 151 151 } {1 3 4} 152 + 153 +# Ticket https://www.sqlite.org/src/info/123c9ba32130a6c9 2017-12-13 154 +# Incorrect result when an idnex is used for an ordered join. 155 +# 156 +# This test case is in the limit2.test module because the problem was first 157 +# exposed by check-in https://www.sqlite.org/src/info/559733b09e which 158 +# implemented the ORDER BY LIMIT optimization that limit2.test strives to 159 +# test. 160 +# 161 +do_execsql_test 600 { 162 + DROP TABLE IF EXISTS t1; 163 + CREATE TABLE t1(a, b); INSERT INTO t1 VALUES(1,2); 164 + DROP TABLE IF EXISTS t2; 165 + CREATE TABLE t2(x, y); INSERT INTO t2 VALUES(1,3); 166 + CREATE INDEX t1ab ON t1(a,b); 167 + SELECT y FROM t1, t2 WHERE a=x AND b<=y ORDER BY b DESC; 168 +} {3} 152 169 153 170 finish_test