Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Bring in the latest enhancements from trunk. |
|---|---|
| Downloads: | Tarball | ZIP archive | SQL archive |
| Timelines: | family | ancestors | descendants | both | apple-osx |
| Files: | files | file ages | folders |
| SHA3-256: |
378230ae7f4b721c8b8d83c8ceb89144 |
| User & Date: | drh 2019-04-15 14:49:49 |
Context
|
2019-05-14
| ||
| 19:39 | Merge recent enhancements from trunk into apple-osx. check-in: 40362d51 user: drh tags: apple-osx | |
|
2019-04-15
| ||
| 14:49 | Bring in the latest enhancements from trunk. check-in: 378230ae user: drh tags: apple-osx | |
| 13:59 | Fix a test script problem in fts5corrupt3.test. check-in: 734192d8 user: dan tags: trunk | |
|
2019-04-04
| ||
| 21:40 | Merge all the latest changes and enhancements from trunk. check-in: 521d5186 user: drh tags: apple-osx | |
Changes
Changes to Makefile.in.
1060 1060 opcodes.h: parse.h $(TOP)/src/vdbe.c $(TOP)/tool/mkopcodeh.tcl 1061 1061 cat parse.h $(TOP)/src/vdbe.c | $(TCLSH_CMD) $(TOP)/tool/mkopcodeh.tcl >opcodes.h 1062 1062 1063 1063 # Rules to build parse.c and parse.h - the outputs of lemon. 1064 1064 # 1065 1065 parse.h: parse.c 1066 1066 1067 -parse.c: $(TOP)/src/parse.y lemon$(BEXE) $(TOP)/tool/addopcodes.tcl 1067 +parse.c: $(TOP)/src/parse.y lemon$(BEXE) 1068 1068 cp $(TOP)/src/parse.y . 1069 - rm -f parse.h 1070 1069 ./lemon$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) parse.y 1071 - mv parse.h parse.h.temp 1072 - $(TCLSH_CMD) $(TOP)/tool/addopcodes.tcl parse.h.temp >parse.h 1073 1070 1074 1071 sqlite3.h: $(TOP)/src/sqlite.h.in $(TOP)/manifest mksourceid$(BEXE) $(TOP)/VERSION 1075 1072 $(TCLSH_CMD) $(TOP)/tool/mksqlite3h.tcl $(TOP) >sqlite3.h 1076 1073 1077 1074 keywordhash.h: $(TOP)/tool/mkkeywordhash.c 1078 1075 $(BCC) -o mkkeywordhash$(BEXE) $(OPT_FEATURE_FLAGS) $(OPTS) $(TOP)/tool/mkkeywordhash.c 1079 1076 ./mkkeywordhash$(BEXE) >keywordhash.h ................................................................................ 1384 1381 $(LTLINK) $(TOP)/tool/loadfts.c libsqlite3.la -o $@ $(TLIBS) 1385 1382 1386 1383 # This target will fail if the SQLite amalgamation contains any exported 1387 1384 # symbols that do not begin with "sqlite3_". It is run as part of the 1388 1385 # releasetest.tcl script. 1389 1386 # 1390 1387 VALIDIDS=' sqlite3(changeset|changegroup|session)?_' 1391 -checksymbols: sqlite3.lo 1392 - nm -g --defined-only sqlite3.lo | egrep -v $(VALIDIDS); test $$? -ne 0 1388 +checksymbols: sqlite3.o 1389 + nm -g --defined-only sqlite3.o 1390 + nm -g --defined-only sqlite3.o | egrep -v $(VALIDIDS); test $$? -ne 0 1393 1391 echo '0 errors out of 1 tests' 1394 1392 1395 1393 # Build the amalgamation-autoconf package. The amalamgation-tarball target builds 1396 1394 # a tarball named for the version number. Ex: sqlite-autoconf-3110000.tar.gz. 1397 1395 # The snapshot-tarball target builds a tarball named by the SHA1 hash 1398 1396 # 1399 1397 amalgamation-tarball: sqlite3.c
Changes to Makefile.msc.
2135 2135 opcodes.h: parse.h $(TOP)\src\vdbe.c $(TOP)\tool\mkopcodeh.tcl 2136 2136 type parse.h $(TOP)\src\vdbe.c | $(TCLSH_CMD) $(TOP)\tool\mkopcodeh.tcl > opcodes.h 2137 2137 2138 2138 # Rules to build parse.c and parse.h - the outputs of lemon. 2139 2139 # 2140 2140 parse.h: parse.c 2141 2141 2142 -parse.c: $(TOP)\src\parse.y lemon.exe $(TOP)\tool\addopcodes.tcl 2142 +parse.c: $(TOP)\src\parse.y lemon.exe 2143 2143 del /Q parse.y parse.h parse.h.temp 2>NUL 2144 2144 copy $(TOP)\src\parse.y . 2145 2145 .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) parse.y 2146 - move parse.h parse.h.temp 2147 - $(TCLSH_CMD) $(TOP)\tool\addopcodes.tcl parse.h.temp > parse.h 2148 2146 2149 2147 $(SQLITE3H): $(TOP)\src\sqlite.h.in $(TOP)\manifest mksourceid.exe $(TOP)\VERSION 2150 2148 $(TCLSH_CMD) $(TOP)\tool\mksqlite3h.tcl $(TOP:\=/) > $(SQLITE3H) $(MKSQLITE3H_ARGS) 2151 2149 2152 2150 sqlite3ext.h: .target_source 2153 2151 !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 2154 2152 type tsrc\sqlite3ext.h | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*\)" "(SQLITE_CALLBACK *)" \
Changes to README.md.
171 171 used to generate that documentation are in a separate source repository. 172 172 173 173 The SQL language parser is **parse.c** which is generate from a grammar in 174 174 the src/parse.y file. The conversion of "parse.y" into "parse.c" is done 175 175 by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code 176 176 for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a 177 177 template for generating its parser. 178 - 179 178 Lemon also generates the **parse.h** header file, at the same time it 180 -generates parse.c. But the parse.h header file is 181 -modified further (to add additional symbols) using the ./addopcodes.tcl 182 -Tcl script. 179 +generates parse.c. 183 180 184 181 The **opcodes.h** header file contains macros that define the numbers 185 182 corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h 186 183 file is generated by the scanning the src/vdbe.c source file. The 187 184 Tcl script at ./mkopcodeh.tcl does this scan and generates opcodes.h. 188 185 A second Tcl script, ./mkopcodec.tcl, then scans opcodes.h to generate 189 186 the **opcodes.c** source file, which contains a reverse mapping from
Changes to ext/fts3/fts3_snippet.c.
124 124 /************************************************************************* 125 125 ** Start of MatchinfoBuffer code. 126 126 */ 127 127 128 128 /* 129 129 ** Allocate a two-slot MatchinfoBuffer object. 130 130 */ 131 -static MatchinfoBuffer *fts3MIBufferNew(int nElem, const char *zMatchinfo){ 131 +static MatchinfoBuffer *fts3MIBufferNew(size_t nElem, const char *zMatchinfo){ 132 132 MatchinfoBuffer *pRet; 133 - int nByte = sizeof(u32) * (2*nElem + 1) + sizeof(MatchinfoBuffer); 134 - int nStr = (int)strlen(zMatchinfo); 133 + sqlite3_int64 nByte = sizeof(u32) * (2*(sqlite3_int64)nElem + 1) 134 + + sizeof(MatchinfoBuffer); 135 + sqlite3_int64 nStr = strlen(zMatchinfo); 135 136 136 - pRet = sqlite3_malloc(nByte + nStr+1); 137 + pRet = sqlite3_malloc64(nByte + nStr+1); 137 138 if( pRet ){ 138 139 memset(pRet, 0, nByte); 139 140 pRet->aMatchinfo[0] = (u8*)(&pRet->aMatchinfo[1]) - (u8*)pRet; 140 - pRet->aMatchinfo[1+nElem] = pRet->aMatchinfo[0] + sizeof(u32)*(nElem+1); 141 - pRet->nElem = nElem; 141 + pRet->aMatchinfo[1+nElem] = pRet->aMatchinfo[0] 142 + + sizeof(u32)*((int)nElem+1); 143 + pRet->nElem = (int)nElem; 142 144 pRet->zMatchinfo = ((char*)pRet) + nByte; 143 145 memcpy(pRet->zMatchinfo, zMatchinfo, nStr+1); 144 146 pRet->aRef[0] = 1; 145 147 } 146 148 147 149 return pRet; 148 150 } ................................................................................ 995 997 ){ 996 998 return SQLITE_OK; 997 999 } 998 1000 sqlite3Fts3ErrMsg(pzErr, "unrecognized matchinfo request: %c", cArg); 999 1001 return SQLITE_ERROR; 1000 1002 } 1001 1003 1002 -static int fts3MatchinfoSize(MatchInfo *pInfo, char cArg){ 1003 - int nVal; /* Number of integers output by cArg */ 1004 +static size_t fts3MatchinfoSize(MatchInfo *pInfo, char cArg){ 1005 + size_t nVal; /* Number of integers output by cArg */ 1004 1006 1005 1007 switch( cArg ){ 1006 1008 case FTS3_MATCHINFO_NDOC: 1007 1009 case FTS3_MATCHINFO_NPHRASE: 1008 1010 case FTS3_MATCHINFO_NCOL: 1009 1011 nVal = 1; 1010 1012 break; ................................................................................ 1280 1282 if( rc==SQLITE_OK ){ 1281 1283 rc = fts3MatchinfoLcs(pCsr, pInfo); 1282 1284 } 1283 1285 break; 1284 1286 1285 1287 case FTS3_MATCHINFO_LHITS_BM: 1286 1288 case FTS3_MATCHINFO_LHITS: { 1287 - int nZero = fts3MatchinfoSize(pInfo, zArg[i]) * sizeof(u32); 1289 + size_t nZero = fts3MatchinfoSize(pInfo, zArg[i]) * sizeof(u32); 1288 1290 memset(pInfo->aMatchinfo, 0, nZero); 1289 1291 rc = fts3ExprLHitGather(pCsr->pExpr, pInfo); 1290 1292 break; 1291 1293 } 1292 1294 1293 1295 default: { 1294 1296 Fts3Expr *pExpr; ................................................................................ 1349 1351 1350 1352 /* If Fts3Cursor.pMIBuffer is NULL, then this is the first time the 1351 1353 ** matchinfo function has been called for this query. In this case 1352 1354 ** allocate the array used to accumulate the matchinfo data and 1353 1355 ** initialize those elements that are constant for every row. 1354 1356 */ 1355 1357 if( pCsr->pMIBuffer==0 ){ 1356 - int nMatchinfo = 0; /* Number of u32 elements in match-info */ 1358 + size_t nMatchinfo = 0; /* Number of u32 elements in match-info */ 1357 1359 int i; /* Used to iterate through zArg */ 1358 1360 1359 1361 /* Determine the number of phrases in the query */ 1360 1362 pCsr->nPhrase = fts3ExprPhraseCount(pCsr->pExpr); 1361 1363 sInfo.nPhrase = pCsr->nPhrase; 1362 1364 1363 1365 /* Determine the number of integers in the buffer returned by this call. */
Changes to ext/fts3/fts3_test.c.
444 444 while( p<pEnd && testIsTokenChar(*p)==0 ) p++; 445 445 446 446 if( p==pEnd ){ 447 447 rc = SQLITE_DONE; 448 448 }else{ 449 449 /* Advance to the end of the token */ 450 450 const char *pToken = p; 451 - int nToken; 451 + sqlite3_int64 nToken; 452 452 while( p<pEnd && testIsTokenChar(*p) ) p++; 453 - nToken = (int)(p-pToken); 453 + nToken = (sqlite3_int64)(p-pToken); 454 454 455 455 /* Copy the token into the buffer */ 456 456 if( nToken>pCsr->nBuffer ){ 457 457 sqlite3_free(pCsr->aBuffer); 458 - pCsr->aBuffer = sqlite3_malloc(nToken); 458 + pCsr->aBuffer = sqlite3_malloc64(nToken); 459 459 } 460 460 if( pCsr->aBuffer==0 ){ 461 461 rc = SQLITE_NOMEM; 462 462 }else{ 463 463 int i; 464 464 465 465 if( pCsr->iLangid & 0x00000001 ){ ................................................................................ 467 467 }else{ 468 468 for(i=0; i<nToken; i++) pCsr->aBuffer[i] = (char)testTolower(pToken[i]); 469 469 } 470 470 pCsr->iToken++; 471 471 pCsr->iInput = (int)(p - pCsr->aInput); 472 472 473 473 *ppToken = pCsr->aBuffer; 474 - *pnBytes = nToken; 474 + *pnBytes = (int)nToken; 475 475 *piStartOffset = (int)(pToken - pCsr->aInput); 476 476 *piEndOffset = (int)(p - pCsr->aInput); 477 477 *piPosition = pCsr->iToken; 478 478 } 479 479 } 480 480 481 481 return rc;
Changes to ext/fts3/fts3_tokenize_vtab.c.
342 342 UNUSED_PARAMETER(idxStr); 343 343 UNUSED_PARAMETER(nVal); 344 344 345 345 fts3tokResetCursor(pCsr); 346 346 if( idxNum==1 ){ 347 347 const char *zByte = (const char *)sqlite3_value_text(apVal[0]); 348 348 int nByte = sqlite3_value_bytes(apVal[0]); 349 - pCsr->zInput = sqlite3_malloc(nByte+1); 349 + pCsr->zInput = sqlite3_malloc64(nByte+1); 350 350 if( pCsr->zInput==0 ){ 351 351 rc = SQLITE_NOMEM; 352 352 }else{ 353 353 memcpy(pCsr->zInput, zByte, nByte); 354 354 pCsr->zInput[nByte] = 0; 355 355 rc = pTab->pMod->xOpen(pTab->pTok, pCsr->zInput, nByte, &pCsr->pCsr); 356 356 if( rc==SQLITE_OK ){
Changes to ext/fts3/fts3_tokenizer.c.
192 192 sqlite3Fts3ErrMsg(pzErr, "unknown tokenizer: %s", z); 193 193 rc = SQLITE_ERROR; 194 194 }else{ 195 195 char const **aArg = 0; 196 196 int iArg = 0; 197 197 z = &z[n+1]; 198 198 while( z<zEnd && (NULL!=(z = (char *)sqlite3Fts3NextToken(z, &n))) ){ 199 - int nNew = sizeof(char *)*(iArg+1); 200 - char const **aNew = (const char **)sqlite3_realloc((void *)aArg, nNew); 199 + sqlite3_int64 nNew = sizeof(char *)*(iArg+1); 200 + char const **aNew = (const char **)sqlite3_realloc64((void *)aArg, nNew); 201 201 if( !aNew ){ 202 202 sqlite3_free(zCopy); 203 203 sqlite3_free((void *)aArg); 204 204 return SQLITE_NOMEM; 205 205 } 206 206 aArg = aNew; 207 207 aArg[iArg++] = z;
Changes to ext/fts3/fts3_write.c.
1748 1748 if( pE ){ 1749 1749 aElem = &pE; 1750 1750 nElem = 1; 1751 1751 } 1752 1752 } 1753 1753 1754 1754 if( nElem>0 ){ 1755 - int nByte = sizeof(Fts3SegReader) + (nElem+1)*sizeof(Fts3HashElem *); 1756 - pReader = (Fts3SegReader *)sqlite3_malloc(nByte); 1755 + sqlite3_int64 nByte; 1756 + nByte = sizeof(Fts3SegReader) + (nElem+1)*sizeof(Fts3HashElem *); 1757 + pReader = (Fts3SegReader *)sqlite3_malloc64(nByte); 1757 1758 if( !pReader ){ 1758 1759 rc = SQLITE_NOMEM; 1759 1760 }else{ 1760 1761 memset(pReader, 0, nByte); 1761 1762 pReader->iIdx = 0x7FFFFFFF; 1762 1763 pReader->ppNextElem = (Fts3HashElem **)&pReader[1]; 1763 1764 memcpy(pReader->ppNextElem, aElem, nElem*sizeof(Fts3HashElem *)); ................................................................................ 3363 3364 ){ 3364 3365 char *pBlob; /* The BLOB encoding of the document size */ 3365 3366 int nBlob; /* Number of bytes in the BLOB */ 3366 3367 sqlite3_stmt *pStmt; /* Statement used to insert the encoding */ 3367 3368 int rc; /* Result code from subfunctions */ 3368 3369 3369 3370 if( *pRC ) return; 3370 - pBlob = sqlite3_malloc( 10*p->nColumn ); 3371 + pBlob = sqlite3_malloc64( 10*(sqlite3_int64)p->nColumn ); 3371 3372 if( pBlob==0 ){ 3372 3373 *pRC = SQLITE_NOMEM; 3373 3374 return; 3374 3375 } 3375 3376 fts3EncodeIntArray(p->nColumn, aSz, pBlob, &nBlob); 3376 3377 rc = fts3SqlStmt(p, SQL_REPLACE_DOCSIZE, &pStmt, 0); 3377 3378 if( rc ){ ................................................................................ 3413 3414 sqlite3_stmt *pStmt; /* Statement for reading and writing */ 3414 3415 int i; /* Loop counter */ 3415 3416 int rc; /* Result code from subfunctions */ 3416 3417 3417 3418 const int nStat = p->nColumn+2; 3418 3419 3419 3420 if( *pRC ) return; 3420 - a = sqlite3_malloc( (sizeof(u32)+10)*nStat ); 3421 + a = sqlite3_malloc64( (sizeof(u32)+10)*(sqlite3_int64)nStat ); 3421 3422 if( a==0 ){ 3422 3423 *pRC = SQLITE_NOMEM; 3423 3424 return; 3424 3425 } 3425 3426 pBlob = (char*)&a[nStat]; 3426 3427 rc = fts3SqlStmt(p, SQL_SELECT_STAT, &pStmt, 0); 3427 3428 if( rc ){ ................................................................................ 3534 3535 rc = SQLITE_NOMEM; 3535 3536 }else{ 3536 3537 rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0); 3537 3538 sqlite3_free(zSql); 3538 3539 } 3539 3540 3540 3541 if( rc==SQLITE_OK ){ 3541 - int nByte = sizeof(u32) * (p->nColumn+1)*3; 3542 - aSz = (u32 *)sqlite3_malloc(nByte); 3542 + sqlite3_int64 nByte = sizeof(u32) * ((sqlite3_int64)p->nColumn+1)*3; 3543 + aSz = (u32 *)sqlite3_malloc64(nByte); 3543 3544 if( aSz==0 ){ 3544 3545 rc = SQLITE_NOMEM; 3545 3546 }else{ 3546 3547 memset(aSz, 0, nByte); 3547 3548 aSzIns = &aSz[p->nColumn+1]; 3548 3549 aSzDel = &aSzIns[p->nColumn+1]; 3549 3550 } ................................................................................ 3601 3602 Fts3Table *p, /* FTS3 table handle */ 3602 3603 sqlite3_int64 iAbsLevel, /* Absolute level to open */ 3603 3604 int nSeg, /* Number of segments to merge */ 3604 3605 Fts3MultiSegReader *pCsr /* Cursor object to populate */ 3605 3606 ){ 3606 3607 int rc; /* Return Code */ 3607 3608 sqlite3_stmt *pStmt = 0; /* Statement used to read %_segdir entry */ 3608 - int nByte; /* Bytes allocated at pCsr->apSegment[] */ 3609 + sqlite3_int64 nByte; /* Bytes allocated at pCsr->apSegment[] */ 3609 3610 3610 3611 /* Allocate space for the Fts3MultiSegReader.aCsr[] array */ 3611 3612 memset(pCsr, 0, sizeof(*pCsr)); 3612 3613 nByte = sizeof(Fts3SegReader *) * nSeg; 3613 - pCsr->apSegment = (Fts3SegReader **)sqlite3_malloc(nByte); 3614 + pCsr->apSegment = (Fts3SegReader **)sqlite3_malloc64(nByte); 3614 3615 3615 3616 if( pCsr->apSegment==0 ){ 3616 3617 rc = SQLITE_NOMEM; 3617 3618 }else{ 3618 3619 memset(pCsr->apSegment, 0, nByte); 3619 3620 rc = fts3SqlStmt(p, SQL_SELECT_LEVEL, &pStmt, 0); 3620 3621 } ................................................................................ 5586 5587 5587 5588 if( nArg>1 && sqlite3_value_int(apVal[2 + p->nColumn + 2])<0 ){ 5588 5589 rc = SQLITE_CONSTRAINT; 5589 5590 goto update_out; 5590 5591 } 5591 5592 5592 5593 /* Allocate space to hold the change in document sizes */ 5593 - aSzDel = sqlite3_malloc( sizeof(aSzDel[0])*(p->nColumn+1)*2 ); 5594 + aSzDel = sqlite3_malloc64(sizeof(aSzDel[0])*((sqlite3_int64)p->nColumn+1)*2); 5594 5595 if( aSzDel==0 ){ 5595 5596 rc = SQLITE_NOMEM; 5596 5597 goto update_out; 5597 5598 } 5598 5599 aSzIns = &aSzDel[p->nColumn+1]; 5599 5600 memset(aSzDel, 0, sizeof(aSzDel[0])*(p->nColumn+1)*2); 5600 5601
Changes to ext/fts5/fts5_index.c.
2639 2639 if( p1->pLeaf==0 ){ /* If p1 is at EOF */ 2640 2640 iRes = i2; 2641 2641 }else if( p2->pLeaf==0 ){ /* If p2 is at EOF */ 2642 2642 iRes = i1; 2643 2643 }else{ 2644 2644 int res = fts5BufferCompare(&p1->term, &p2->term); 2645 2645 if( res==0 ){ 2646 - assert( i2>i1 ); 2647 - assert( i2!=0 ); 2646 + assert_nc( i2>i1 ); 2647 + assert_nc( i2!=0 ); 2648 2648 pRes->bTermEq = 1; 2649 2649 if( p1->iRowid==p2->iRowid ){ 2650 2650 p1->bDel = p2->bDel; 2651 2651 return i2; 2652 2652 } 2653 2653 res = ((p1->iRowid > p2->iRowid)==pIter->bRev) ? -1 : +1; 2654 2654 } ................................................................................ 3687 3687 if( p->rc==SQLITE_OK && nLvl>=pWriter->nDlidx ){ 3688 3688 Fts5DlidxWriter *aDlidx = (Fts5DlidxWriter*)sqlite3_realloc64( 3689 3689 pWriter->aDlidx, sizeof(Fts5DlidxWriter) * nLvl 3690 3690 ); 3691 3691 if( aDlidx==0 ){ 3692 3692 p->rc = SQLITE_NOMEM; 3693 3693 }else{ 3694 - int nByte = sizeof(Fts5DlidxWriter) * (nLvl - pWriter->nDlidx); 3694 + size_t nByte = sizeof(Fts5DlidxWriter) * (nLvl - pWriter->nDlidx); 3695 3695 memset(&aDlidx[pWriter->nDlidx], 0, nByte); 3696 3696 pWriter->aDlidx = aDlidx; 3697 3697 pWriter->nDlidx = nLvl; 3698 3698 } 3699 3699 } 3700 3700 return p->rc; 3701 3701 }
Changes to ext/fts5/fts5_main.c.
2464 2464 fts5_extension_function xFunc, /* Aux. function implementation */ 2465 2465 void(*xDestroy)(void*) /* Destructor for pUserData */ 2466 2466 ){ 2467 2467 Fts5Global *pGlobal = (Fts5Global*)pApi; 2468 2468 int rc = sqlite3_overload_function(pGlobal->db, zName, -1); 2469 2469 if( rc==SQLITE_OK ){ 2470 2470 Fts5Auxiliary *pAux; 2471 - int nName; /* Size of zName in bytes, including \0 */ 2472 - int nByte; /* Bytes of space to allocate */ 2471 + sqlite3_int64 nName; /* Size of zName in bytes, including \0 */ 2472 + sqlite3_int64 nByte; /* Bytes of space to allocate */ 2473 2473 2474 - nName = (int)strlen(zName) + 1; 2474 + nName = strlen(zName) + 1; 2475 2475 nByte = sizeof(Fts5Auxiliary) + nName; 2476 - pAux = (Fts5Auxiliary*)sqlite3_malloc(nByte); 2476 + pAux = (Fts5Auxiliary*)sqlite3_malloc64(nByte); 2477 2477 if( pAux ){ 2478 - memset(pAux, 0, nByte); 2478 + memset(pAux, 0, (size_t)nByte); 2479 2479 pAux->zFunc = (char*)&pAux[1]; 2480 2480 memcpy(pAux->zFunc, zName, nName); 2481 2481 pAux->pGlobal = pGlobal; 2482 2482 pAux->pUserData = pUserData; 2483 2483 pAux->xFunc = xFunc; 2484 2484 pAux->xDestroy = xDestroy; 2485 2485 pAux->pNext = pGlobal->pAux; ................................................................................ 2501 2501 const char *zName, /* Name of new function */ 2502 2502 void *pUserData, /* User data for aux. function */ 2503 2503 fts5_tokenizer *pTokenizer, /* Tokenizer implementation */ 2504 2504 void(*xDestroy)(void*) /* Destructor for pUserData */ 2505 2505 ){ 2506 2506 Fts5Global *pGlobal = (Fts5Global*)pApi; 2507 2507 Fts5TokenizerModule *pNew; 2508 - int nName; /* Size of zName and its \0 terminator */ 2509 - int nByte; /* Bytes of space to allocate */ 2508 + sqlite3_int64 nName; /* Size of zName and its \0 terminator */ 2509 + sqlite3_int64 nByte; /* Bytes of space to allocate */ 2510 2510 int rc = SQLITE_OK; 2511 2511 2512 - nName = (int)strlen(zName) + 1; 2512 + nName = strlen(zName) + 1; 2513 2513 nByte = sizeof(Fts5TokenizerModule) + nName; 2514 - pNew = (Fts5TokenizerModule*)sqlite3_malloc(nByte); 2514 + pNew = (Fts5TokenizerModule*)sqlite3_malloc64(nByte); 2515 2515 if( pNew ){ 2516 - memset(pNew, 0, nByte); 2516 + memset(pNew, 0, (size_t)nByte); 2517 2517 pNew->zName = (char*)&pNew[1]; 2518 2518 memcpy(pNew->zName, zName, nName); 2519 2519 pNew->pUserData = pUserData; 2520 2520 pNew->x = *pTokenizer; 2521 2521 pNew->xDestroy = xDestroy; 2522 2522 pNew->pNext = pGlobal->pTok; 2523 2523 pGlobal->pTok = pNew;
Changes to ext/fts5/fts5_tokenize.c.
365 365 if( p ){ 366 366 const char *zCat = "L* N* Co"; 367 367 int i; 368 368 memset(p, 0, sizeof(Unicode61Tokenizer)); 369 369 370 370 p->eRemoveDiacritic = FTS5_REMOVE_DIACRITICS_SIMPLE; 371 371 p->nFold = 64; 372 - p->aFold = sqlite3_malloc(p->nFold * sizeof(char)); 372 + p->aFold = sqlite3_malloc64(p->nFold * sizeof(char)); 373 373 if( p->aFold==0 ){ 374 374 rc = SQLITE_NOMEM; 375 375 } 376 376 377 377 /* Search for a "categories" argument */ 378 378 for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ 379 379 if( 0==sqlite3_stricmp(azArg[i], "categories") ){
Changes to ext/fts5/test/fts5corrupt3.test.
7990 7990 | 4080: 67 73 7a 18 0b 03 1b 01 76 65 72 73 69 6f 6e 04 gsz.....version. 7991 7991 | page 6 offset 20480 7992 7992 | 0: 0d 00 00 00 03 0f f2 00 0f fc 0f f7 0f f2 00 00 ................ 7993 7993 | 4080: 00 00 03 03 02 01 03 03 02 02 01 02 02 01 02 09 ................ 7994 7994 | end crash-2acc487d09f033.db 7995 7995 }]} {} 7996 7996 7997 -do_catchsql_test 56.1 { 7998 - INSERT INTO t1(b) VALUES(randomblob(250)); 7999 - INSERT INTO t1(b) VALUES(randomblob(250)); 7997 +do_test 56.1 { 7998 + set res [catchsql { 7999 + INSERT INTO t1(b) VALUES(randomblob(250)); 8000 + INSERT INTO t1(b) VALUES(randomblob(250)); 8001 + }] 8002 + 8003 + # For some permutations - those that use the page-cache - this test 8004 + # may return SQLITE_CONSTRAINT instead of SQLITE_CORRUPT. This is because 8005 + # the corrupt db in the test over-reads the page buffer slightly, with 8006 + # different results depending on whether or not the page-cache is in use. 8007 + if {$res=="1 {constraint failed}"} { 8008 + set res "1 {database disk image is malformed}" 8009 + } 8010 + set res 8011 +} {1 {database disk image is malformed}} 8012 + 8013 +#------------------------------------------------------------------------- 8014 +reset_db 8015 +do_test 57.0 { 8016 + sqlite3 db {} 8017 + db deserialize [decode_hexdb { 8018 +| size 28672 pagesize 4096 filename x.db 8019 +| page 1 offset 0 8020 +| 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. 8021 +| 16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 07 .....@ ........ 8022 +| 32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ 8023 +| 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ................ 8024 +| 96: 00 2e 34 20 0d 00 00 00 07 0d d2 00 0f c4 0f 6d ..4 ...........m 8025 +| 112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00 .....N.......... 8026 +| 3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet 8027 +| 3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE 8028 +| 3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta 8029 +| 3584: 61 6b 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 aket1_configt1_c 8030 +| 3600: 6f 7e 66 69 67 06 43 52 45 41 54 45 20 54 41 42 o~fig.CREATE TAB 8031 +| 3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k 8032 +| 3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v) 8033 +| 3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05 WITHOUT ROWID[. 8034 +| 3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d 8035 +| 3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize 8036 +| 3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't 8037 +| 3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN 8038 +| 3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE 8039 +| 3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21 Y, sz BLOB)U...! 8040 +| 3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 1d !.wtablet1_cont. 8041 +| 3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45 ntt1_content.CRE 8042 +| 3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f ATE TABLE 't1_co 8043 +| 3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45 ntent'(id INTEGE 8044 +| 3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63 R PRIMARY KEY, c 8045 +| 3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 0)i.......-table 8046 +| 3856: 74 31 5f 69 64 78 74 31 5f 59 64 78 03 43 52 45 t1_idxt1_Ydx.CRE 8047 +| 3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 ATE TABLE 't1_id 8048 +| 3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 x'(segid, term, 8049 +| 3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 pgno, PRIMARY KE 8050 +| 3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 Y(segid, term)) 8051 +| 3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 WITHOUT ROWIDU.. 8052 +| 3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 ......tablet1_da 8053 +| 3968: 74 61 74 31 5f 64 61 64 61 02 43 52 45 41 54 45 tat1_dada.CREATE 8054 +| 3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 TABLE 't1_data' 8055 +| 4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d (id INTEGER PRIM 8056 +| 4016: 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 ARY KEY, block B 8057 +| 4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c LOB):......ctabl 8058 +| 4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54 et1t1CREATE VIRT 8059 +| 4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49 UAL TABLE t1 USI 8060 +| 4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29 NG fts5(content) 8061 +| page 2 offset 4096 8062 +| 0: 0d 0e b4 00 06 0e 35 00 0f e8 0e 35 0f bd 0f 4e ......5....5...N 8063 +| 16: 0e cb 0e 4f 00 00 00 00 00 00 00 00 00 00 00 00 ...O............ 8064 +| 3632: 00 00 00 00 00 18 0a 03 00 36 00 00 00 00 01 04 .........6...... 8065 +| 3648: 04 00 04 01 01 01 02 01 01 03 01 01 04 01 01 5e ...............^ 8066 +| 3664: 90 80 80 80 80 01 04 00 81 40 00 00 00 51 06 30 .........@...Q.0 8067 +| 3680: 61 62 61 63 6b 01 01 04 04 6e 64 6f 6e 01 01 02 aback....ndon... 8068 +| 3696: 04 63 69 76 65 01 01 02 04 6c 70 68 61 01 01 02 .cive....lpha... 8069 +| 3712: 03 74 6f 6d 01 01 01 06 62 61 63 6b 75 70 01 01 .tom....backup.. 8070 +| 3728: 02 05 6f 6f 6d 65 72 01 01 01 06 63 68 61 6e 6e ..oomer....chann 8071 +| 3744: 65 01 01 01 04 74 65 73 74 01 01 04 09 08 08 08 e....test....... 8072 +| 3760: 07 0a 09 0a 0f 3a 00 17 30 00 00 00 00 01 03 03 .....:..0....... 8073 +| 3776: 00 03 01 01 01 02 01 01 03 01 01 68 8c 80 80 80 ...........h.... 8074 +| 3792: 80 01 04 00 81 54 00 00 00 5b 06 30 61 62 61 63 .....T...[.0abac 8075 +| 3808: 6b 02 02 07 04 04 6e 64 6f 6e 02 02 05 02 04 63 k.....ndon.....c 8076 +| 3824: 69 76 65 02 02 0b 02 04 6c 70 68 61 02 04 02 0a ive.....lpha.... 8077 +| 3840: 02 03 74 6f 6d 02 02 09 01 06 62 61 63 6b 75 70 ..tom.....backup 8078 +| 3856: 02 02 04 02 05 6f 6f 6d 65 72 02 02 08 01 06 63 .....oomer.....c 8079 +| 3872: 68 61 6e 6e 65 02 02 03 01 04 74 65 73 74 02 02 hanne.....test.. 8080 +| 3888: 06 04 0a 09 09 0a 08 0b 0a 0b 0f ef 00 14 2a 00 ..............*. 8081 +| 3904: 00 00 00 01 02 02 00 02 01 01 01 02 01 01 68 88 ..............h. 8082 +| 3920: 80 80 80 80 01 04 00 81 54 00 00 00 5b 06 30 61 ........T...[.0a 8083 +| 3936: 62 61 63 6b 01 02 07 04 04 6e 64 6f 6e 01 02 05 back.....ndon... 8084 +| 3952: 02 04 63 69 76 65 01 02 0b 02 04 6c 70 68 61 01 ..cive.....lpha. 8085 +| 3968: 04 02 0a 02 03 74 6f 6d 01 02 09 01 06 62 61 63 .....tom.....bac 8086 +| 3984: 6b 75 70 01 02 04 02 05 6f 6f 6d 65 72 01 02 08 kup.....oomer... 8087 +| 4000: 01 06 63 68 61 6e 6e 65 01 02 03 01 04 74 65 73 ..channe.....tes 8088 +| 4016: 74 01 02 06 04 0a 09 09 0a 08 0b 0a 0b 24 84 80 t............$.. 8089 +| 4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61 ......N.....0aba 8090 +| 4048: 63 6b 01 02 02 05 42 66 74 02 02 02 04 04 6e 64 ck....Bft.....nd 8091 +| 4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 04 0d 00 on.............. 8092 +| 4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ 8093 +| page 3 offset 8192 8094 +| 0: 0a 00 00 00 04 0f e5 00 00 00 0f f3 0f ec 0f e5 ................ 8095 +| 4064: 00 00 00 00 00 06 04 01 0c 01 04 02 06 04 01 0c ................ 8096 +| 4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0d 01 02 ................ 8097 +| page 4 offset 12288 8098 +| 0: 0d 0e bc 00 04 0e 78 00 00 00 00 00 00 00 0e 78 ......x........x 8099 +| 16: 0e 78 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .x.............. 8100 +| 3696: 00 00 00 00 00 00 00 00 42 02 04 00 81 09 61 6c ........B.....al 8101 +| 3712: 70 68 61 20 63 68 61 6e 6e 65 20 62 61 63 6b 75 pha channe backu 8102 +| 3728: 70 20 61 62 61 6e 64 6f 6e 20 74 65 73 74 20 61 p abandon test a 8103 +| 3744: 62 61 63 6b 20 62 6f 6f 6d 65 72 20 61 74 6f 6d back boomer atom 8104 +| 3760: 20 61 6c 70 68 61 20 61 63 69 76 65 00 00 00 44 alpha acive...D 8105 +| 3776: 81 09 61 6c 70 68 61 20 63 68 61 6e 6e 65 20 62 ..alpha channe b 8106 +| 3792: 61 63 6b 75 70 20 61 62 61 6e 64 6f 6e 20 74 65 ackup abandon te 8107 +| 3808: 73 74 20 61 62 61 63 6b 20 62 6f 6f 6d 65 72 20 st aback boomer 8108 +| 3824: 61 74 6f 6d 20 61 6c 70 68 61 20 61 63 69 76 65 atom alpha acive 8109 +| 4064: 0a 03 03 00 1b 61 4e 61 6e 64 6f 6e 08 02 03 00 .....aNandon.... 8110 +| 4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 71 63 6b .abaft.....abqck 8111 +| page 5 offset 16384 8112 +| 0: 0d 0f e8 00 04 0f e2 00 00 00 00 00 00 00 0f e2 ................ 8113 +| 16: 0f e2 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 8114 +| 4064: 00 00 04 02 03 00 0e 0a 00 00 00 06 0e 0a 04 03 ................ 8115 +| 4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 10 0e 01 ................ 8116 +| page 6 offset 20480 8117 +| 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ 8118 +| 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. 8119 +| page 7 offset 24576 8120 +| 0: 0d 00 00 00 03 0f d6 00 0f f4 00 00 00 00 00 00 ................ 8121 +| 4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c ..........rebuil 8122 +| 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c 8123 +| 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize 8124 +| end x.db 8125 +}]} {} 8126 + 8127 +do_catchsql_test 57.1 { 8128 + INSERT INTO t1(t1) VALUES('optimize') 8000 8129 } {1 {database disk image is malformed}} 8130 + 8001 8131 8002 8132 sqlite3_fts5_may_be_corrupt 0 8003 8133 finish_test 8004 8134
Added ext/rbu/rbupartial.test.
1 +# 2019 April 11 2 +# 3 +# The author disclaims copyright to this source code. In place of 4 +# a legal notice, here is a blessing: 5 +# 6 +# May you do good and not evil. 7 +# May you find forgiveness for yourself and forgive others. 8 +# May you share freely, never taking more than you give. 9 +# 10 +#*********************************************************************** 11 +# 12 + 13 +source [file join [file dirname [info script]] rbu_common.tcl] 14 +set ::testprefix rbupartial 15 + 16 +db close 17 +sqlite3_shutdown 18 +sqlite3_config_uri 1 19 + 20 +foreach {tn without_rowid a b c d} { 21 + 1 "" a b c d 22 + 2 "WITHOUT ROWID" aaa bbb ccc ddd 23 + 3 "WITHOUT ROWID" "\"hello\"" {"one'two"} {[c]} ddd 24 + 4 "WITHOUT ROWID" {`a b`} {"one'two"} {[c c c]} ddd 25 + 5 "" a b c {"d""d"} 26 + 6 "" {'one''two'} b {"c""c"} {"d""d"} 27 +} { 28 + eval [string map [list \ 29 + %WITHOUT_ROWID% $without_rowid %A% $a %B% $b %C% $c %D% $d 30 + ] { 31 + reset_db 32 + do_execsql_test $tn.1.0 { 33 + CREATE TABLE t1(%A% PRIMARY KEY, %B%, %C%, %D%) %WITHOUT_ROWID% ; 34 + CREATE INDEX i1b ON t1(%B%); 35 + CREATE INDEX i1b2 ON t1(%B%) WHERE %C%<5; 36 + CREATE INDEX i1b3 ON t1(%B%) WHERE %C%>=5; 37 + 38 + CREATE INDEX i1c ON t1(%C%); 39 + CREATE INDEX i1c2 ON t1(%C%) WHERE %C% IS NULL; 40 + CREATE INDEX i1c3 ON t1(%C%) WHERE %C% IS NOT NULL; 41 + 42 + CREATE INDEX i1c4 ON t1(%C%) WHERE %D% < 'd'; 43 + } 44 + 45 + do_execsql_test $tn.1.1 { 46 + INSERT INTO t1 VALUES(0, NULL, NULL, 'a'); 47 + INSERT INTO t1 VALUES(1, 2, 3, 'b'); 48 + INSERT INTO t1 VALUES(4, 5, 6, 'c'); 49 + INSERT INTO t1 VALUES(7, 8, 9, 'd'); 50 + } 51 + 52 + forcedelete rbu.db 53 + do_test $tn.1.2 { 54 + sqlite3 rbu rbu.db 55 + rbu eval { 56 + CREATE TABLE data_t1(%A%, %B%, %C%, %D%, rbu_control); 57 + 58 + INSERT INTO data_t1 VALUES(10, 11, 12, 'e', 0); 59 + INSERT INTO data_t1 VALUES(13, 14, NULL, 'f', 0); 60 + 61 + INSERT INTO data_t1 VALUES(0, NULL, NULL, NULL, 1); 62 + INSERT INTO data_t1 VALUES(4, NULL, NULL, NULL, 1); 63 + 64 + INSERT INTO data_t1 VALUES(7, NULL, 4, NULL, '..x.'); 65 + INSERT INTO data_t1 VALUES(1, 10, NULL, NULL, '.xx.'); 66 + } 67 + rbu close 68 + } {} 69 + 70 + do_test $tn.1.3 { 71 + run_rbu test.db rbu.db 72 + execsql { PRAGMA integrity_check } 73 + } {ok} 74 + 75 + do_execsql_test $tn.1.4 { 76 + SELECT * FROM t1 ORDER BY %A%; 77 + } { 78 + 1 10 {} b 7 8 4 d 10 11 12 e 13 14 {} f 79 + } 80 + 81 + set step 0 82 + do_rbu_vacuum_test $tn.1.5 0 83 + }] 84 +} 85 + 86 +finish_test
Changes to ext/rbu/sqlite3rbu.c.
236 236 ** * a special "cleanup table" state. 237 237 ** 238 238 ** abIndexed: 239 239 ** If the table has no indexes on it, abIndexed is set to NULL. Otherwise, 240 240 ** it points to an array of flags nTblCol elements in size. The flag is 241 241 ** set for each column that is either a part of the PK or a part of an 242 242 ** index. Or clear otherwise. 243 +** 244 +** If there are one or more partial indexes on the table, all fields of 245 +** this array set set to 1. This is because in that case, the module has 246 +** no way to tell which fields will be required to add and remove entries 247 +** from the partial indexes. 243 248 ** 244 249 */ 245 250 struct RbuObjIter { 246 251 sqlite3_stmt *pTblIter; /* Iterate through tables */ 247 252 sqlite3_stmt *pIdxIter; /* Index iterator */ 248 253 int nTblCol; /* Size of azTblCol[] array */ 249 254 char **azTblCol; /* Array of unquoted target column names */ ................................................................................ 1031 1036 ** 1032 1037 ** If an error (i.e. an OOM condition) occurs, return NULL and leave an 1033 1038 ** error code in the rbu handle passed as the first argument. Or, if an 1034 1039 ** error has already occurred when this function is called, return NULL 1035 1040 ** immediately without attempting the allocation or modifying the stored 1036 1041 ** error code. 1037 1042 */ 1038 -static void *rbuMalloc(sqlite3rbu *p, int nByte){ 1043 +static void *rbuMalloc(sqlite3rbu *p, sqlite3_int64 nByte){ 1039 1044 void *pRet = 0; 1040 1045 if( p->rc==SQLITE_OK ){ 1041 1046 assert( nByte>0 ); 1042 1047 pRet = sqlite3_malloc64(nByte); 1043 1048 if( pRet==0 ){ 1044 1049 p->rc = SQLITE_NOMEM; 1045 1050 }else{ ................................................................................ 1052 1057 1053 1058 /* 1054 1059 ** Allocate and zero the pIter->azTblCol[] and abTblPk[] arrays so that 1055 1060 ** there is room for at least nCol elements. If an OOM occurs, store an 1056 1061 ** error code in the RBU handle passed as the first argument. 1057 1062 */ 1058 1063 static void rbuAllocateIterArrays(sqlite3rbu *p, RbuObjIter *pIter, int nCol){ 1059 - int nByte = (2*sizeof(char*) + sizeof(int) + 3*sizeof(u8)) * nCol; 1064 + sqlite3_int64 nByte = (2*sizeof(char*) + sizeof(int) + 3*sizeof(u8)) * nCol; 1060 1065 char **azNew; 1061 1066 1062 1067 azNew = (char**)rbuMalloc(p, nByte); 1063 1068 if( azNew ){ 1064 1069 pIter->azTblCol = azNew; 1065 1070 pIter->azTblType = &azNew[nCol]; 1066 1071 pIter->aiSrcOrder = (int*)&pIter->azTblType[nCol]; ................................................................................ 1246 1251 sqlite3_mprintf("PRAGMA main.index_list = %Q", pIter->zTbl) 1247 1252 ); 1248 1253 } 1249 1254 1250 1255 pIter->nIndex = 0; 1251 1256 while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pList) ){ 1252 1257 const char *zIdx = (const char*)sqlite3_column_text(pList, 1); 1258 + int bPartial = sqlite3_column_int(pList, 4); 1253 1259 sqlite3_stmt *pXInfo = 0; 1254 1260 if( zIdx==0 ) break; 1261 + if( bPartial ){ 1262 + memset(pIter->abIndexed, 0x01, sizeof(u8)*pIter->nTblCol); 1263 + } 1255 1264 p->rc = prepareFreeAndCollectError(p->dbMain, &pXInfo, &p->zErrmsg, 1256 1265 sqlite3_mprintf("PRAGMA main.index_xinfo = %Q", zIdx) 1257 1266 ); 1258 1267 while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXInfo) ){ 1259 1268 int iCid = sqlite3_column_int(pXInfo, 1); 1260 1269 if( iCid>=0 ) pIter->abIndexed[iCid] = 1; 1261 1270 } ................................................................................ 1692 1701 ** string, an error code is left in the rbu handle passed as the first 1693 1702 ** argument and NULL is returned. Or, if an error has already occurred 1694 1703 ** when this function is called, NULL is returned immediately, without 1695 1704 ** attempting the allocation or modifying the stored error code. 1696 1705 */ 1697 1706 static char *rbuObjIterGetBindlist(sqlite3rbu *p, int nBind){ 1698 1707 char *zRet = 0; 1699 - int nByte = nBind*2 + 1; 1708 + sqlite3_int64 nByte = 2*(sqlite3_int64)nBind + 1; 1700 1709 1701 1710 zRet = (char*)rbuMalloc(p, nByte); 1702 1711 if( zRet ){ 1703 1712 int i; 1704 1713 for(i=0; i<nBind; i++){ 1705 1714 zRet[i*2] = '?'; 1706 1715 zRet[i*2+1] = (i+1==nBind) ? '\0' : ','; ................................................................................ 1953 1962 rc = sqlite3_reset(p->objiter.pTmpInsert); 1954 1963 } 1955 1964 1956 1965 if( rc!=SQLITE_OK ){ 1957 1966 sqlite3_result_error_code(pCtx, rc); 1958 1967 } 1959 1968 } 1969 + 1970 +static char *rbuObjIterGetIndexWhere(sqlite3rbu *p, RbuObjIter *pIter){ 1971 + sqlite3_stmt *pStmt = 0; 1972 + int rc = p->rc; 1973 + char *zRet = 0; 1974 + 1975 + if( rc==SQLITE_OK ){ 1976 + rc = prepareAndCollectError(p->dbMain, &pStmt, &p->zErrmsg, 1977 + "SELECT trim(sql) FROM sqlite_master WHERE type='index' AND name=?" 1978 + ); 1979 + } 1980 + if( rc==SQLITE_OK ){ 1981 + int rc2; 1982 + rc = sqlite3_bind_text(pStmt, 1, pIter->zIdx, -1, SQLITE_STATIC); 1983 + if( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){ 1984 + const char *zSql = (const char*)sqlite3_column_text(pStmt, 0); 1985 + if( zSql ){ 1986 + int nParen = 0; /* Number of open parenthesis */ 1987 + int i; 1988 + for(i=0; zSql[i]; i++){ 1989 + char c = zSql[i]; 1990 + if( c=='(' ){ 1991 + nParen++; 1992 + } 1993 + else if( c==')' ){ 1994 + nParen--; 1995 + if( nParen==0 ){ 1996 + i++; 1997 + break; 1998 + } 1999 + }else if( c=='"' || c=='\'' || c=='`' ){ 2000 + for(i++; 1; i++){ 2001 + if( zSql[i]==c ){ 2002 + if( zSql[i+1]!=c ) break; 2003 + i++; 2004 + } 2005 + } 2006 + }else if( c=='[' ){ 2007 + for(i++; 1; i++){ 2008 + if( zSql[i]==']' ) break; 2009 + } 2010 + } 2011 + } 2012 + if( zSql[i] ){ 2013 + zRet = rbuStrndup(&zSql[i], &rc); 2014 + } 2015 + } 2016 + } 2017 + 2018 + rc2 = sqlite3_finalize(pStmt); 2019 + if( rc==SQLITE_OK ) rc = rc2; 2020 + } 2021 + 2022 + p->rc = rc; 2023 + return zRet; 2024 +} 1960 2025 1961 2026 /* 1962 2027 ** Ensure that the SQLite statement handles required to update the 1963 2028 ** target database object currently indicated by the iterator passed 1964 2029 ** as the second argument are available. 1965 2030 */ 1966 2031 static int rbuObjIterPrepareAll( ................................................................................ 1983 2048 1984 2049 if( zIdx ){ 1985 2050 const char *zTbl = pIter->zTbl; 1986 2051 char *zImposterCols = 0; /* Columns for imposter table */ 1987 2052 char *zImposterPK = 0; /* Primary key declaration for imposter */ 1988 2053 char *zWhere = 0; /* WHERE clause on PK columns */ 1989 2054 char *zBind = 0; 2055 + char *zPart = 0; 1990 2056 int nBind = 0; 1991 2057 1992 2058 assert( pIter->eType!=RBU_PK_VTAB ); 1993 2059 zCollist = rbuObjIterGetIndexCols( 1994 2060 p, pIter, &zImposterCols, &zImposterPK, &zWhere, &nBind 1995 2061 ); 1996 2062 zBind = rbuObjIterGetBindlist(p, nBind); 2063 + zPart = rbuObjIterGetIndexWhere(p, pIter); 1997 2064 1998 2065 /* Create the imposter table used to write to this index. */ 1999 2066 sqlite3_test_control(SQLITE_TESTCTRL_IMPOSTER, p->dbMain, "main", 0, 1); 2000 2067 sqlite3_test_control(SQLITE_TESTCTRL_IMPOSTER, p->dbMain, "main", 1,tnum); 2001 2068 rbuMPrintfExec(p, p->dbMain, 2002 2069 "CREATE TABLE \"rbu_imp_%w\"( %s, PRIMARY KEY( %s ) ) WITHOUT ROWID", 2003 2070 zTbl, zImposterCols, zImposterPK ................................................................................ 2022 2089 } 2023 2090 2024 2091 /* Create the SELECT statement to read keys in sorted order */ 2025 2092 if( p->rc==SQLITE_OK ){ 2026 2093 char *zSql; 2027 2094 if( rbuIsVacuum(p) ){ 2028 2095 zSql = sqlite3_mprintf( 2029 - "SELECT %s, 0 AS rbu_control FROM '%q' ORDER BY %s%s", 2096 + "SELECT %s, 0 AS rbu_control FROM '%q' %s ORDER BY %s%s", 2030 2097 zCollist, 2031 2098 pIter->zDataTbl, 2032 - zCollist, zLimit 2099 + zPart, zCollist, zLimit 2033 2100 ); 2034 2101 }else 2035 2102 2036 2103 if( pIter->eType==RBU_PK_EXTERNAL || pIter->eType==RBU_PK_NONE ){ 2037 2104 zSql = sqlite3_mprintf( 2038 - "SELECT %s, rbu_control FROM %s.'rbu_tmp_%q' ORDER BY %s%s", 2105 + "SELECT %s, rbu_control FROM %s.'rbu_tmp_%q' %s ORDER BY %s%s", 2039 2106 zCollist, p->zStateDb, pIter->zDataTbl, 2040 - zCollist, zLimit 2107 + zPart, zCollist, zLimit 2041 2108 ); 2042 2109 }else{ 2043 2110 zSql = sqlite3_mprintf( 2044 - "SELECT %s, rbu_control FROM %s.'rbu_tmp_%q' " 2111 + "SELECT %s, rbu_control FROM %s.'rbu_tmp_%q' %s " 2045 2112 "UNION ALL " 2046 2113 "SELECT %s, rbu_control FROM '%q' " 2047 - "WHERE typeof(rbu_control)='integer' AND rbu_control!=1 " 2114 + "%s %s typeof(rbu_control)='integer' AND rbu_control!=1 " 2048 2115 "ORDER BY %s%s", 2049 - zCollist, p->zStateDb, pIter->zDataTbl, 2116 + zCollist, p->zStateDb, pIter->zDataTbl, zPart, 2050 2117 zCollist, pIter->zDataTbl, 2118 + zPart, 2119 + (zPart ? "AND" : "WHERE"), 2051 2120 zCollist, zLimit 2052 2121 ); 2053 2122 } 2054 2123 p->rc = prepareFreeAndCollectError(p->dbRbu, &pIter->pSelect, pz, zSql); 2055 2124 } 2056 2125 2057 2126 sqlite3_free(zImposterCols); 2058 2127 sqlite3_free(zImposterPK); 2059 2128 sqlite3_free(zWhere); 2060 2129 sqlite3_free(zBind); 2130 + sqlite3_free(zPart); 2061 2131 }else{ 2062 2132 int bRbuRowid = (pIter->eType==RBU_PK_VTAB) 2063 2133 ||(pIter->eType==RBU_PK_NONE) 2064 2134 ||(pIter->eType==RBU_PK_EXTERNAL && rbuIsVacuum(p)); 2065 2135 const char *zTbl = pIter->zTbl; /* Table this step applies to */ 2066 2136 const char *zWrite; /* Imposter table name */ 2067 2137 ................................................................................ 4487 4557 4488 4558 /* If not in RBU_STAGE_OAL, allow this call to pass through. Or, if this 4489 4559 ** rbu is in the RBU_STAGE_OAL state, use heap memory for *-shm space 4490 4560 ** instead of a file on disk. */ 4491 4561 assert( p->openFlags & (SQLITE_OPEN_MAIN_DB|SQLITE_OPEN_TEMP_DB) ); 4492 4562 if( eStage==RBU_STAGE_OAL || eStage==RBU_STAGE_MOVE ){ 4493 4563 if( iRegion<=p->nShm ){ 4494 - int nByte = (iRegion+1) * sizeof(char*); 4564 + sqlite3_int64 nByte = (iRegion+1) * sizeof(char*); 4495 4565 char **apNew = (char**)sqlite3_realloc64(p->apShm, nByte); 4496 4566 if( apNew==0 ){ 4497 4567 rc = SQLITE_NOMEM; 4498 4568 }else{ 4499 4569 memset(&apNew[p->nShm], 0, sizeof(char*) * (1 + iRegion - p->nShm)); 4500 4570 p->apShm = apNew; 4501 4571 p->nShm = iRegion+1;
Changes to ext/rtree/geopoly.c.
265 265 && s.a[0]==s.a[s.nVertex*2-2] 266 266 && s.a[1]==s.a[s.nVertex*2-1] 267 267 && (s.z++, geopolySkipSpace(&s)==0) 268 268 ){ 269 269 GeoPoly *pOut; 270 270 int x = 1; 271 271 s.nVertex--; /* Remove the redundant vertex at the end */ 272 - pOut = sqlite3_malloc64( GEOPOLY_SZ(s.nVertex) ); 272 + pOut = sqlite3_malloc64( GEOPOLY_SZ((sqlite3_int64)s.nVertex) ); 273 273 x = 1; 274 274 if( pOut==0 ) goto parse_json_err; 275 275 pOut->nVertex = s.nVertex; 276 276 memcpy(pOut->a, s.a, s.nVertex*2*sizeof(GeoCoord)); 277 277 pOut->hdr[0] = *(unsigned char*)&x; 278 278 pOut->hdr[1] = (s.nVertex>>16)&0xff; 279 279 pOut->hdr[2] = (s.nVertex>>8)&0xff; ................................................................................ 651 651 r = GeoY(p,ii); 652 652 if( r<mnY ) mnY = (float)r; 653 653 else if( r>mxY ) mxY = (float)r; 654 654 } 655 655 if( pRc ) *pRc = SQLITE_OK; 656 656 if( aCoord==0 ){ 657 657 geopolyBboxFill: 658 - pOut = sqlite3_realloc(p, GEOPOLY_SZ(4)); 658 + pOut = sqlite3_realloc64(p, GEOPOLY_SZ(4)); 659 659 if( pOut==0 ){ 660 660 sqlite3_free(p); 661 661 if( context ) sqlite3_result_error_nomem(context); 662 662 if( pRc ) *pRc = SQLITE_NOMEM; 663 663 return 0; 664 664 } 665 665 pOut->nVertex = 4; ................................................................................ 1047 1047 return p; 1048 1048 } 1049 1049 1050 1050 /* 1051 1051 ** Determine the overlap between two polygons 1052 1052 */ 1053 1053 static int geopolyOverlap(GeoPoly *p1, GeoPoly *p2){ 1054 - int nVertex = p1->nVertex + p2->nVertex + 2; 1054 + sqlite3_int64 nVertex = p1->nVertex + p2->nVertex + 2; 1055 1055 GeoOverlap *p; 1056 - int nByte; 1056 + sqlite3_int64 nByte; 1057 1057 GeoEvent *pThisEvent; 1058 1058 double rX; 1059 1059 int rc = 0; 1060 1060 int needSort = 0; 1061 1061 GeoSegment *pActive = 0; 1062 1062 GeoSegment *pSeg; 1063 1063 unsigned char aOverlap[4]; 1064 1064 1065 1065 nByte = sizeof(GeoEvent)*nVertex*2 1066 1066 + sizeof(GeoSegment)*nVertex 1067 1067 + sizeof(GeoOverlap); 1068 - p = sqlite3_malloc( nByte ); 1068 + p = sqlite3_malloc64( nByte ); 1069 1069 if( p==0 ) return -1; 1070 1070 p->aEvent = (GeoEvent*)&p[1]; 1071 1071 p->aSegment = (GeoSegment*)&p->aEvent[nVertex*2]; 1072 1072 p->nEvent = p->nSegment = 0; 1073 1073 geopolyAddSegments(p, p1, 1); 1074 1074 geopolyAddSegments(p, p2, 2); 1075 1075 pThisEvent = geopolySortEventsByX(p->aEvent, p->nEvent); ................................................................................ 1220 1220 int argc, const char *const*argv, /* Parameters to CREATE TABLE statement */ 1221 1221 sqlite3_vtab **ppVtab, /* OUT: New virtual table */ 1222 1222 char **pzErr, /* OUT: Error message, if any */ 1223 1223 int isCreate /* True for xCreate, false for xConnect */ 1224 1224 ){ 1225 1225 int rc = SQLITE_OK; 1226 1226 Rtree *pRtree; 1227 - int nDb; /* Length of string argv[1] */ 1228 - int nName; /* Length of string argv[2] */ 1227 + sqlite3_int64 nDb; /* Length of string argv[1] */ 1228 + sqlite3_int64 nName; /* Length of string argv[2] */ 1229 1229 sqlite3_str *pSql; 1230 1230 char *zSql; 1231 1231 int ii; 1232 1232 1233 1233 sqlite3_vtab_config(db, SQLITE_VTAB_CONSTRAINT_SUPPORT, 1); 1234 1234 1235 1235 /* Allocate the sqlite3_vtab structure */ 1236 - nDb = (int)strlen(argv[1]); 1237 - nName = (int)strlen(argv[2]); 1238 - pRtree = (Rtree *)sqlite3_malloc(sizeof(Rtree)+nDb+nName+2); 1236 + nDb = strlen(argv[1]); 1237 + nName = strlen(argv[2]); 1238 + pRtree = (Rtree *)sqlite3_malloc64(sizeof(Rtree)+nDb+nName+2); 1239 1239 if( !pRtree ){ 1240 1240 return SQLITE_NOMEM; 1241 1241 } 1242 1242 memset(pRtree, 0, sizeof(Rtree)+nDb+nName+2); 1243 1243 pRtree->nBusy = 1; 1244 1244 pRtree->base.pModule = &rtreeModule; 1245 1245 pRtree->zDb = (char *)&pRtree[1];
Changes to ext/session/sqlite3session.c.
898 898 ** Growing the hash table in this case is a performance optimization only, 899 899 ** it is not required for correct operation. 900 900 */ 901 901 static int sessionGrowHash(int bPatchset, SessionTable *pTab){ 902 902 if( pTab->nChange==0 || pTab->nEntry>=(pTab->nChange/2) ){ 903 903 int i; 904 904 SessionChange **apNew; 905 - int nNew = (pTab->nChange ? pTab->nChange : 128) * 2; 905 + sqlite3_int64 nNew = 2*(sqlite3_int64)(pTab->nChange ? pTab->nChange : 128); 906 906 907 907 apNew = (SessionChange **)sqlite3_malloc64(sizeof(SessionChange *) * nNew); 908 908 if( apNew==0 ){ 909 909 if( pTab->nChange==0 ){ 910 910 return SQLITE_ERROR; 911 911 } 912 912 return SQLITE_OK; ................................................................................ 1825 1825 /* 1826 1826 ** Ensure that there is room in the buffer to append nByte bytes of data. 1827 1827 ** If not, use sqlite3_realloc() to grow the buffer so that there is. 1828 1828 ** 1829 1829 ** If successful, return zero. Otherwise, if an OOM condition is encountered, 1830 1830 ** set *pRc to SQLITE_NOMEM and return non-zero. 1831 1831 */ 1832 -static int sessionBufferGrow(SessionBuffer *p, int nByte, int *pRc){ 1832 +static int sessionBufferGrow(SessionBuffer *p, size_t nByte, int *pRc){ 1833 1833 if( *pRc==SQLITE_OK && p->nAlloc-p->nBuf<nByte ){ 1834 1834 u8 *aNew; 1835 1835 i64 nNew = p->nAlloc ? p->nAlloc : 128; 1836 1836 do { 1837 1837 nNew = nNew*2; 1838 1838 }while( (nNew-p->nBuf)<nByte ); 1839 1839 ................................................................................ 2943 2943 sessionBufferGrow(&p->tblhdr, nByte, &rc); 2944 2944 }else{ 2945 2945 rc = SQLITE_CORRUPT_BKPT; 2946 2946 } 2947 2947 } 2948 2948 2949 2949 if( rc==SQLITE_OK ){ 2950 - int iPK = sizeof(sqlite3_value*)*p->nCol*2; 2950 + size_t iPK = sizeof(sqlite3_value*)*p->nCol*2; 2951 2951 memset(p->tblhdr.aBuf, 0, iPK); 2952 2952 memcpy(&p->tblhdr.aBuf[iPK], &p->in.aData[p->in.iNext], nCopy); 2953 2953 p->in.iNext += nCopy; 2954 2954 } 2955 2955 2956 2956 p->apValue = (sqlite3_value**)p->tblhdr.aBuf; 2957 2957 p->abPK = (u8*)&p->apValue[p->nCol*2]; ................................................................................ 4247 4247 while( pApply->constraints.nBuf ){ 4248 4248 sqlite3_changeset_iter *pIter2 = 0; 4249 4249 SessionBuffer cons = pApply->constraints; 4250 4250 memset(&pApply->constraints, 0, sizeof(SessionBuffer)); 4251 4251 4252 4252 rc = sessionChangesetStart(&pIter2, 0, 0, cons.nBuf, cons.aBuf, 0); 4253 4253 if( rc==SQLITE_OK ){ 4254 - int nByte = 2*pApply->nCol*sizeof(sqlite3_value*); 4254 + size_t nByte = 2*pApply->nCol*sizeof(sqlite3_value*); 4255 4255 int rc2; 4256 4256 pIter2->bPatchset = bPatchset; 4257 4257 pIter2->zTab = (char*)zTab; 4258 4258 pIter2->nCol = pApply->nCol; 4259 4259 pIter2->abPK = pApply->abPK; 4260 4260 sessionBufferGrow(&pIter2->tblhdr, nByte, &rc); 4261 4261 pIter2->apValue = (sqlite3_value**)pIter2->tblhdr.aBuf;
Changes to main.mk.
377 377 $(TOP)/ext/misc/regexp.c \ 378 378 $(TOP)/ext/misc/remember.c \ 379 379 $(TOP)/ext/misc/series.c \ 380 380 $(TOP)/ext/misc/spellfix.c \ 381 381 $(TOP)/ext/misc/totype.c \ 382 382 $(TOP)/ext/misc/unionvtab.c \ 383 383 $(TOP)/ext/misc/wholenumber.c \ 384 - $(TOP)/ext/misc/vfslog.c \ 385 384 $(TOP)/ext/misc/zipfile.c \ 386 385 $(TOP)/ext/fts5/fts5_tcl.c \ 387 386 $(TOP)/ext/fts5/fts5_test_mi.c \ 388 387 $(TOP)/ext/fts5/fts5_test_tok.c 389 388 390 389 391 390 #TESTSRC += $(TOP)/ext/fts2/fts2_tokenizer.c ................................................................................ 717 716 cat parse.h $(TOP)/src/vdbe.c | \ 718 717 tclsh $(TOP)/tool/mkopcodeh.tcl >opcodes.h 719 718 720 719 # Rules to build parse.c and parse.h - the outputs of lemon. 721 720 # 722 721 parse.h: parse.c 723 722 724 -parse.c: $(TOP)/src/parse.y lemon $(TOP)/tool/addopcodes.tcl 723 +parse.c: $(TOP)/src/parse.y lemon 725 724 cp $(TOP)/src/parse.y . 726 - rm -f parse.h 727 725 ./lemon -s $(OPTS) parse.y 728 - mv parse.h parse.h.temp 729 - tclsh $(TOP)/tool/addopcodes.tcl parse.h.temp >parse.h 730 726 731 727 sqlite3.h: $(TOP)/src/sqlite.h.in $(TOP)/manifest mksourceid $(TOP)/VERSION $(TOP)/ext/rtree/sqlite3rtree.h 732 728 tclsh $(TOP)/tool/mksqlite3h.tcl $(TOP) >sqlite3.h 733 729 734 730 keywordhash.h: $(TOP)/tool/mkkeywordhash.c 735 731 $(BCC) -o mkkeywordhash $(OPTS) $(TOP)/tool/mkkeywordhash.c 736 732 ./mkkeywordhash >keywordhash.h
Changes to src/btree.c.
6172 6172 */ 6173 6173 static int freePage2(BtShared *pBt, MemPage *pMemPage, Pgno iPage){ 6174 6174 MemPage *pTrunk = 0; /* Free-list trunk page */ 6175 6175 Pgno iTrunk = 0; /* Page number of free-list trunk page */ 6176 6176 MemPage *pPage1 = pBt->pPage1; /* Local reference to page 1 */ 6177 6177 MemPage *pPage; /* Page being freed. May be NULL. */ 6178 6178 int rc; /* Return Code */ 6179 - int nFree; /* Initial number of pages on free-list */ 6179 + u32 nFree; /* Initial number of pages on free-list */ 6180 6180 6181 6181 assert( sqlite3_mutex_held(pBt->mutex) ); 6182 6182 assert( CORRUPT_DB || iPage>1 ); 6183 6183 assert( !pMemPage || pMemPage->pgno==iPage ); 6184 6184 6185 6185 if( iPage<2 || iPage>pBt->nPage ){ 6186 6186 return SQLITE_CORRUPT_BKPT;
Changes to src/build.c.
3764 3764 sqlite3 *db, /* Connection to notify of malloc failures */ 3765 3765 void *pArray, /* Array of objects. Might be reallocated */ 3766 3766 int szEntry, /* Size of each object in the array */ 3767 3767 int *pnEntry, /* Number of objects currently in use */ 3768 3768 int *pIdx /* Write the index of a new slot here */ 3769 3769 ){ 3770 3770 char *z; 3771 - int n = *pnEntry; 3771 + sqlite3_int64 n = *pIdx = *pnEntry; 3772 3772 if( (n & (n-1))==0 ){ 3773 - int sz = (n==0) ? 1 : 2*n; 3773 + sqlite3_int64 sz = (n==0) ? 1 : 2*n; 3774 3774 void *pNew = sqlite3DbRealloc(db, pArray, sz*szEntry); 3775 3775 if( pNew==0 ){ 3776 3776 *pIdx = -1; 3777 3777 return pArray; 3778 3778 } 3779 3779 pArray = pNew; 3780 3780 } 3781 3781 z = (char*)pArray; 3782 3782 memset(&z[n * szEntry], 0, szEntry); 3783 - *pIdx = n; 3784 3783 ++*pnEntry; 3785 3784 return pArray; 3786 3785 } 3787 3786 3788 3787 /* 3789 3788 ** Append a new element to the given IdList. Create a new IdList if 3790 3789 ** need be. ................................................................................ 3887 3886 assert( nExtra>=1 ); 3888 3887 assert( pSrc!=0 ); 3889 3888 assert( iStart<=pSrc->nSrc ); 3890 3889 3891 3890 /* Allocate additional space if needed */ 3892 3891 if( (u32)pSrc->nSrc+nExtra>pSrc->nAlloc ){ 3893 3892 SrcList *pNew; 3894 - int nAlloc = pSrc->nSrc*2+nExtra; 3893 + sqlite3_int64 nAlloc = 2*(sqlite3_int64)pSrc->nSrc+nExtra; 3895 3894 sqlite3 *db = pParse->db; 3896 3895 3897 3896 if( pSrc->nSrc+nExtra>=SQLITE_MAX_SRCLIST ){ 3898 3897 sqlite3ErrorMsg(pParse, "too many FROM clause terms, max: %d", 3899 3898 SQLITE_MAX_SRCLIST); 3900 3899 return 0; 3901 3900 } ................................................................................ 4394 4393 Index *pIdx /* The index that triggers the constraint */ 4395 4394 ){ 4396 4395 char *zErr; 4397 4396 int j; 4398 4397 StrAccum errMsg; 4399 4398 Table *pTab = pIdx->pTable; 4400 4399 4401 - sqlite3StrAccumInit(&errMsg, pParse->db, 0, 0, 200); 4400 + sqlite3StrAccumInit(&errMsg, pParse->db, 0, 0, 4401 + pParse->db->aLimit[SQLITE_LIMIT_LENGTH]); 4402 4402 if( pIdx->aColExpr ){ 4403 4403 sqlite3_str_appendf(&errMsg, "index '%q'", pIdx->zName); 4404 4404 }else{ 4405 4405 for(j=0; j<pIdx->nKeyCol; j++){ 4406 4406 char *zCol; 4407 4407 assert( pIdx->aiColumn[j]>=0 ); 4408 4408 zCol = pTab->aCol[pIdx->aiColumn[j]].zName; ................................................................................ 4643 4643 if( sqlite3StrICmp(zName, pWith->a[i].zName)==0 ){ 4644 4644 sqlite3ErrorMsg(pParse, "duplicate WITH table name: %s", zName); 4645 4645 } 4646 4646 } 4647 4647 } 4648 4648 4649 4649 if( pWith ){ 4650 - int nByte = sizeof(*pWith) + (sizeof(pWith->a[1]) * pWith->nCte); 4650 + sqlite3_int64 nByte = sizeof(*pWith) + (sizeof(pWith->a[1]) * pWith->nCte); 4651 4651 pNew = sqlite3DbRealloc(db, pWith, nByte); 4652 4652 }else{ 4653 4653 pNew = sqlite3DbMallocZero(db, sizeof(*pWith)); 4654 4654 } 4655 4655 assert( (pNew!=0 && zName!=0) || db->mallocFailed ); 4656 4656 4657 4657 if( db->mallocFailed ){
Changes to src/expr.c.
853 853 if( op==TK_AND && pParse->nErr==0 && !IN_RENAME_OBJECT ){ 854 854 /* Take advantage of short-circuit false optimization for AND */ 855 855 p = sqlite3ExprAnd(pParse->db, pLeft, pRight); 856 856 }else{ 857 857 p = sqlite3DbMallocRawNN(pParse->db, sizeof(Expr)); 858 858 if( p ){ 859 859 memset(p, 0, sizeof(Expr)); 860 - p->op = op & TKFLG_MASK; 860 + p->op = op & 0xff; 861 861 p->iAgg = -1; 862 862 } 863 863 sqlite3ExprAttachSubtrees(pParse->db, p, pLeft, pRight); 864 864 } 865 865 if( p ) { 866 866 sqlite3ExprCheckHeight(pParse, p->nHeight); 867 867 } ................................................................................ 1318 1318 ** argument. If an OOM condition is encountered, NULL is returned 1319 1319 ** and the db->mallocFailed flag set. 1320 1320 */ 1321 1321 #ifndef SQLITE_OMIT_CTE 1322 1322 static With *withDup(sqlite3 *db, With *p){ 1323 1323 With *pRet = 0; 1324 1324 if( p ){ 1325 - int nByte = sizeof(*p) + sizeof(p->a[0]) * (p->nCte-1); 1325 + sqlite3_int64 nByte = sizeof(*p) + sizeof(p->a[0]) * (p->nCte-1); 1326 1326 pRet = sqlite3DbMallocZero(db, nByte); 1327 1327 if( pRet ){ 1328 1328 int i; 1329 1329 pRet->nCte = p->nCte; 1330 1330 for(i=0; i<p->nCte; i++){ 1331 1331 pRet->a[i].pSelect = sqlite3SelectDup(db, p->a[i].pSelect, 0); 1332 1332 pRet->a[i].pCols = sqlite3ExprListDup(db, p->a[i].pCols, 0); ................................................................................ 1583 1583 if( pList==0 ){ 1584 1584 goto no_mem; 1585 1585 } 1586 1586 pList->nExpr = 0; 1587 1587 }else if( (pList->nExpr & (pList->nExpr-1))==0 ){ 1588 1588 ExprList *pNew; 1589 1589 pNew = sqlite3DbRealloc(db, pList, 1590 - sizeof(*pList)+(2*pList->nExpr - 1)*sizeof(pList->a[0])); 1590 + sizeof(*pList)+(2*(sqlite3_int64)pList->nExpr-1)*sizeof(pList->a[0])); 1591 1591 if( pNew==0 ){ 1592 1592 goto no_mem; 1593 1593 } 1594 1594 pList = pNew; 1595 1595 } 1596 1596 pItem = &pList->a[pList->nExpr++]; 1597 1597 assert( offsetof(struct ExprList_item,zName)==sizeof(pItem->pExpr) );
Changes to src/hash.c.
146 146 */ 147 147 static HashElem *findElementWithHash( 148 148 const Hash *pH, /* The pH to be searched */ 149 149 const char *pKey, /* The key we are searching for */ 150 150 unsigned int *pHash /* Write the hash value here */ 151 151 ){ 152 152 HashElem *elem; /* Used to loop thru the element list */ 153 - int count; /* Number of elements left to test */ 153 + unsigned int count; /* Number of elements left to test */ 154 154 unsigned int h; /* The computed hash */ 155 155 static HashElem nullElement = { 0, 0, 0, 0 }; 156 156 157 157 if( pH->ht ){ /*OPTIMIZATION-IF-TRUE*/ 158 158 struct _ht *pEntry; 159 159 h = strHash(pKey) % pH->htsize; 160 160 pEntry = &pH->ht[h]; ................................................................................ 194 194 elem->next->prev = elem->prev; 195 195 } 196 196 if( pH->ht ){ 197 197 pEntry = &pH->ht[h]; 198 198 if( pEntry->chain==elem ){ 199 199 pEntry->chain = elem->next; 200 200 } 201 + assert( pEntry->count>0 ); 201 202 pEntry->count--; 202 - assert( pEntry->count>=0 ); 203 203 } 204 204 sqlite3_free( elem ); 205 205 pH->count--; 206 206 if( pH->count==0 ){ 207 207 assert( pH->first==0 ); 208 208 assert( pH->count==0 ); 209 209 sqlite3HashClear(pH);
Changes to src/hash.h.
41 41 ** the hash table. 42 42 */ 43 43 struct Hash { 44 44 unsigned int htsize; /* Number of buckets in the hash table */ 45 45 unsigned int count; /* Number of entries in this table */ 46 46 HashElem *first; /* The first element of the array */ 47 47 struct _ht { /* the hash table */ 48 - int count; /* Number of entries with this hash */ 48 + unsigned int count; /* Number of entries with this hash */ 49 49 HashElem *chain; /* Pointer to first entry with this hash */ 50 50 } *ht; 51 51 }; 52 52 53 53 /* Each element in the hash table is an instance of the following 54 54 ** structure. All elements are stored on a single doubly-linked list. 55 55 **
Changes to src/insert.c.
2270 2270 destHasUniqueIdx = 1; 2271 2271 } 2272 2272 for(pSrcIdx=pSrc->pIndex; pSrcIdx; pSrcIdx=pSrcIdx->pNext){ 2273 2273 if( xferCompatibleIndex(pDestIdx, pSrcIdx) ) break; 2274 2274 } 2275 2275 if( pSrcIdx==0 ){ 2276 2276 return 0; /* pDestIdx has no corresponding index in pSrc */ 2277 + } 2278 + if( pSrcIdx->tnum==pDestIdx->tnum && pSrc->pSchema==pDest->pSchema 2279 + && sqlite3FaultSim(411)==SQLITE_OK ){ 2280 + /* The sqlite3FaultSim() call allows this corruption test to be 2281 + ** bypassed during testing, in order to exercise other corruption tests 2282 + ** further downstream. */ 2283 + return 0; /* Corrupt schema - two indexes on the same btree */ 2277 2284 } 2278 2285 } 2279 2286 #ifndef SQLITE_OMIT_CHECK 2280 2287 if( pDest->pCheck && sqlite3ExprListCompare(pSrc->pCheck,pDest->pCheck,-1) ){ 2281 2288 return 0; /* Tables have different CHECK constraints. Ticket #2252 */ 2282 2289 } 2283 2290 #endif
Changes to src/main.c.
704 704 if( sz<=(int)sizeof(LookasideSlot*) ) sz = 0; 705 705 if( cnt<0 ) cnt = 0; 706 706 if( sz==0 || cnt==0 ){ 707 707 sz = 0; 708 708 pStart = 0; 709 709 }else if( pBuf==0 ){ 710 710 sqlite3BeginBenignMalloc(); 711 - pStart = sqlite3Malloc( sz*cnt ); /* IMP: R-61949-35727 */ 711 + pStart = sqlite3Malloc( sz*(sqlite3_int64)cnt ); /* IMP: R-61949-35727 */ 712 712 sqlite3EndBenignMalloc(); 713 713 if( pStart ) cnt = sqlite3MallocSize(pStart)/sz; 714 714 }else{ 715 715 pStart = pBuf; 716 716 } 717 717 db->lookaside.pStart = pStart; 718 718 db->lookaside.pInit = 0;
Changes to src/parse.y.
189 189 columnlist ::= columnname carglist. 190 190 columnname(A) ::= nm(A) typetoken(Y). {sqlite3AddColumn(pParse,&A,&Y);} 191 191 192 192 // Declare some tokens early in order to influence their values, to 193 193 // improve performance and reduce the executable size. The goal here is 194 194 // to get the "jump" operations in ISNULL through ESCAPE to have numeric 195 195 // values that are early enough so that all jump operations are clustered 196 -// at the beginning, but also so that the comparison tokens NE through GE 197 -// are as large as possible so that they are near to FUNCTION, which is a 198 -// token synthesized by addopcodes.tcl. 196 +// at the beginning. 199 197 // 200 198 %token ABORT ACTION AFTER ANALYZE ASC ATTACH BEFORE BEGIN BY CASCADE CAST. 201 199 %token CONFLICT DATABASE DEFERRED DESC DETACH EACH END EXCLUSIVE EXPLAIN FAIL. 202 200 %token OR AND NOT IS MATCH LIKE_KW BETWEEN IN ISNULL NOTNULL NE EQ. 203 201 %token GT LE LT GE ESCAPE. 204 202 205 203 // The following directive causes tokens ABORT, AFTER, ASC, etc. to ................................................................................ 1740 1738 sqlite3ExprDelete(pParse->db, W); 1741 1739 } 1742 1740 } 1743 1741 1744 1742 filter_opt(A) ::= . { A = 0; } 1745 1743 filter_opt(A) ::= FILTER LP WHERE expr(X) RP. { A = X; } 1746 1744 %endif /* SQLITE_OMIT_WINDOWFUNC */ 1745 + 1746 +/* 1747 +** The code generator needs some extra TK_ token values for tokens that 1748 +** are synthesized and do not actually appear in the grammar: 1749 +*/ 1750 +%token 1751 + TRUEFALSE /* True or false keyword */ 1752 + ISNOT /* Combination of IS and NOT */ 1753 + FUNCTION /* A function invocation */ 1754 + COLUMN /* Reference to a table column */ 1755 + AGG_FUNCTION /* An aggregate function */ 1756 + AGG_COLUMN /* An aggregated column */ 1757 + UMINUS /* Unary minus */ 1758 + UPLUS /* Unary plus */ 1759 + TRUTH /* IS TRUE or IS FALSE or IS NOT TRUE or IS NOT FALSE */ 1760 + REGISTER /* Reference to a VDBE register */ 1761 + VECTOR /* Vector */ 1762 + SELECT_COLUMN /* Choose a single column from a multi-column SELECT */ 1763 + IF_NULL_ROW /* the if-null-row operator */ 1764 + ASTERISK /* The "*" in count(*) and similar */ 1765 + SPAN /* The span operator */ 1766 +. 1767 +/* There must be no more than 255 tokens defined above. If this grammar 1768 +** is extended with new rules and tokens, they must either be so few in 1769 +** number that TK_SPAN is no more than 255, or else the new tokens must 1770 +** appear after this line. 1771 +*/ 1772 +%include { 1773 +#if TK_SPAN>255 1774 +# error too many tokens in the grammar 1775 +#endif 1776 +} 1777 + 1778 +/* 1779 +** The TK_SPACE and TK_ILLEGAL tokens must be the last two tokens. The 1780 +** parser depends on this. Those tokens are not used in any grammar rule. 1781 +** They are only used by the tokenizer. Declare them last so that they 1782 +** are guaranteed to be the last two tokens 1783 +*/ 1784 +%token SPACE ILLEGAL.
Changes to src/pcache1.c.
484 484 485 485 /* 486 486 ** Malloc function used by SQLite to obtain space from the buffer configured 487 487 ** using sqlite3_config(SQLITE_CONFIG_PAGECACHE) option. If no such buffer 488 488 ** exists, this function falls back to sqlite3Malloc(). 489 489 */ 490 490 void *sqlite3PageMalloc(int sz){ 491 - /* During rebalance operations on a corrupt database file, it is sometimes 492 - ** (rarely) possible to overread the temporary page buffer by a few bytes. 493 - ** Enlarge the allocation slightly so that this does not cause problems. */ 491 + assert( sz<=65536+8 ); /* These allocations are never very large */ 494 492 return pcache1Alloc(sz); 495 493 } 496 494 497 495 /* 498 496 ** Free an allocated buffer obtained from sqlite3PageMalloc(). 499 497 */ 500 498 void sqlite3PageFree(void *p){
Changes to src/printf.c.
133 133 /* 134 134 ** Set the StrAccum object to an error mode. 135 135 */ 136 136 static void setStrAccumError(StrAccum *p, u8 eError){ 137 137 assert( eError==SQLITE_NOMEM || eError==SQLITE_TOOBIG ); 138 138 p->accError = eError; 139 139 if( p->mxAlloc ) sqlite3_str_reset(p); 140 + if( eError==SQLITE_TOOBIG ) sqlite3ErrorToParser(p->db, eError); 140 141 } 141 142 142 143 /* 143 144 ** Extra argument values from a PrintfArguments object 144 145 */ 145 146 static sqlite3_int64 getIntArg(PrintfArguments *p){ 146 147 if( p->nArg<=p->nUsed ) return 0;
Changes to src/sqliteInt.h.
3846 3846 void sqlite3TreeViewWinFunc(TreeView*, const Window*, u8); 3847 3847 #endif 3848 3848 #endif 3849 3849 3850 3850 3851 3851 void sqlite3SetString(char **, sqlite3*, const char*); 3852 3852 void sqlite3ErrorMsg(Parse*, const char*, ...); 3853 +int sqlite3ErrorToParser(sqlite3*,int); 3853 3854 void sqlite3Dequote(char*); 3854 3855 void sqlite3DequoteExpr(Expr*); 3855 3856 void sqlite3TokenInit(Token*,char*); 3856 3857 int sqlite3KeywordCode(const unsigned char*, int); 3857 3858 int sqlite3RunParser(Parse*, const char*, char **); 3858 3859 void sqlite3FinishCoding(Parse*); 3859 3860 int sqlite3GetTempReg(Parse*);
Changes to src/test4.c.
60 60 #define N_THREAD 26 61 61 static Thread threadset[N_THREAD]; 62 62 63 63 64 64 /* 65 65 ** The main loop for a thread. Threads use busy waiting. 66 66 */ 67 -static void *thread_main(void *pArg){ 67 +static void *test_thread_main(void *pArg){ 68 68 Thread *p = (Thread*)pArg; 69 69 if( p->db ){ 70 70 sqlite3_close(p->db); 71 71 } 72 72 sqlite3_open(p->zFilename, &p->db); 73 73 if( SQLITE_OK!=sqlite3_errcode(p->db) ){ 74 74 p->zErr = strdup(sqlite3_errmsg(p->db)); ................................................................................ 147 147 return TCL_ERROR; 148 148 } 149 149 threadset[i].busy = 1; 150 150 sqlite3_free(threadset[i].zFilename); 151 151 threadset[i].zFilename = sqlite3_mprintf("%s", argv[2]); 152 152 threadset[i].opnum = 1; 153 153 threadset[i].completed = 0; 154 - rc = pthread_create(&x, 0, thread_main, &threadset[i]); 154 + rc = pthread_create(&x, 0, test_thread_main, &threadset[i]); 155 155 if( rc ){ 156 156 Tcl_AppendResult(interp, "failed to create the thread", 0); 157 157 sqlite3_free(threadset[i].zFilename); 158 158 threadset[i].busy = 0; 159 159 return TCL_ERROR; 160 160 } 161 161 pthread_detach(x); 162 162 return TCL_OK; 163 163 } 164 164 165 165 /* 166 166 ** Wait for a thread to reach its idle state. 167 167 */ 168 -static void thread_wait(Thread *p){ 168 +static void test_thread_wait(Thread *p){ 169 169 while( p->opnum>p->completed ) sched_yield(); 170 170 } 171 171 172 172 /* 173 173 ** Usage: thread_wait ID 174 174 ** 175 175 ** Wait on thread ID to reach its idle state. ................................................................................ 189 189 } 190 190 i = parse_thread_id(interp, argv[1]); 191 191 if( i<0 ) return TCL_ERROR; 192 192 if( !threadset[i].busy ){ 193 193 Tcl_AppendResult(interp, "no such thread", 0); 194 194 return TCL_ERROR; 195 195 } 196 - thread_wait(&threadset[i]); 196 + test_thread_wait(&threadset[i]); 197 197 return TCL_OK; 198 198 } 199 199 200 200 /* 201 201 ** Stop a thread. 202 202 */ 203 -static void stop_thread(Thread *p){ 204 - thread_wait(p); 203 +static void test_stop_thread(Thread *p){ 204 + test_thread_wait(p); 205 205 p->xOp = 0; 206 206 p->opnum++; 207 - thread_wait(p); 207 + test_thread_wait(p); 208 208 sqlite3_free(p->zArg); 209 209 p->zArg = 0; 210 210 sqlite3_free(p->zFilename); 211 211 p->zFilename = 0; 212 212 p->busy = 0; 213 213 } 214 214 ................................................................................ 229 229 if( argc!=2 ){ 230 230 Tcl_AppendResult(interp, "wrong # args: should be \"", argv[0], 231 231 " ID", 0); 232 232 return TCL_ERROR; 233 233 } 234 234 if( argv[1][0]=='*' && argv[1][1]==0 ){ 235 235 for(i=0; i<N_THREAD; i++){ 236 - if( threadset[i].busy ) stop_thread(&threadset[i]); 236 + if( threadset[i].busy ) test_stop_thread(&threadset[i]); 237 237 } 238 238 }else{ 239 239 i = parse_thread_id(interp, argv[1]); 240 240 if( i<0 ) return TCL_ERROR; 241 241 if( !threadset[i].busy ){ 242 242 Tcl_AppendResult(interp, "no such thread", 0); 243 243 return TCL_ERROR; 244 244 } 245 - stop_thread(&threadset[i]); 245 + test_stop_thread(&threadset[i]); 246 246 } 247 247 return TCL_OK; 248 248 } 249 249 250 250 /* 251 251 ** Usage: thread_argc ID 252 252 ** ................................................................................ 269 269 } 270 270 i = parse_thread_id(interp, argv[1]); 271 271 if( i<0 ) return TCL_ERROR; 272 272 if( !threadset[i].busy ){ 273 273 Tcl_AppendResult(interp, "no such thread", 0); 274 274 return TCL_ERROR; 275 275 } 276 - thread_wait(&threadset[i]); 276 + test_thread_wait(&threadset[i]); 277 277 sqlite3_snprintf(sizeof(zBuf), zBuf, "%d", threadset[i].argc); 278 278 Tcl_AppendResult(interp, zBuf, 0); 279 279 return TCL_OK; 280 280 } 281 281 282 282 /* 283 283 ** Usage: thread_argv ID N ................................................................................ 302 302 i = parse_thread_id(interp, argv[1]); 303 303 if( i<0 ) return TCL_ERROR; 304 304 if( !threadset[i].busy ){ 305 305 Tcl_AppendResult(interp, "no such thread", 0); 306 306 return TCL_ERROR; 307 307 } 308 308 if( Tcl_GetInt(interp, argv[2], &n) ) return TCL_ERROR; 309 - thread_wait(&threadset[i]); 309 + test_thread_wait(&threadset[i]); 310 310 if( n<0 || n>=threadset[i].argc ){ 311 311 Tcl_AppendResult(interp, "column number out of range", 0); 312 312 return TCL_ERROR; 313 313 } 314 314 Tcl_AppendResult(interp, threadset[i].argv[n], 0); 315 315 return TCL_OK; 316 316 } ................................................................................ 338 338 i = parse_thread_id(interp, argv[1]); 339 339 if( i<0 ) return TCL_ERROR; 340 340 if( !threadset[i].busy ){ 341 341 Tcl_AppendResult(interp, "no such thread", 0); 342 342 return TCL_ERROR; 343 343 } 344 344 if( Tcl_GetInt(interp, argv[2], &n) ) return TCL_ERROR; 345 - thread_wait(&threadset[i]); 345 + test_thread_wait(&threadset[i]); 346 346 if( n<0 || n>=threadset[i].argc ){ 347 347 Tcl_AppendResult(interp, "column number out of range", 0); 348 348 return TCL_ERROR; 349 349 } 350 350 Tcl_AppendResult(interp, threadset[i].colv[n], 0); 351 351 return TCL_OK; 352 352 } ................................................................................ 373 373 } 374 374 i = parse_thread_id(interp, argv[1]); 375 375 if( i<0 ) return TCL_ERROR; 376 376 if( !threadset[i].busy ){ 377 377 Tcl_AppendResult(interp, "no such thread", 0); 378 378 return TCL_ERROR; 379 379 } 380 - thread_wait(&threadset[i]); 380 + test_thread_wait(&threadset[i]); 381 381 zName = sqlite3ErrName(threadset[i].rc); 382 382 Tcl_AppendResult(interp, zName, 0); 383 383 return TCL_OK; 384 384 } 385 385 386 386 /* 387 387 ** Usage: thread_error ID ................................................................................ 404 404 } 405 405 i = parse_thread_id(interp, argv[1]); 406 406 if( i<0 ) return TCL_ERROR; 407 407 if( !threadset[i].busy ){ 408 408 Tcl_AppendResult(interp, "no such thread", 0); 409 409 return TCL_ERROR; 410 410 } 411 - thread_wait(&threadset[i]); 411 + test_thread_wait(&threadset[i]); 412 412 Tcl_AppendResult(interp, threadset[i].zErr, 0); 413 413 return TCL_OK; 414 414 } 415 415 416 416 /* 417 417 ** This procedure runs in the thread to compile an SQL statement. 418 418 */ ................................................................................ 448 448 } 449 449 i = parse_thread_id(interp, argv[1]); 450 450 if( i<0 ) return TCL_ERROR; 451 451 if( !threadset[i].busy ){ 452 452 Tcl_AppendResult(interp, "no such thread", 0); 453 453 return TCL_ERROR; 454 454 } 455 - thread_wait(&threadset[i]); 455 + test_thread_wait(&threadset[i]); 456 456 threadset[i].xOp = do_compile; 457 457 sqlite3_free(threadset[i].zArg); 458 458 threadset[i].zArg = sqlite3_mprintf("%s", argv[2]); 459 459 threadset[i].opnum++; 460 460 return TCL_OK; 461 461 } 462 462 ................................................................................ 501 501 } 502 502 i = parse_thread_id(interp, argv[1]); 503 503 if( i<0 ) return TCL_ERROR; 504 504 if( !threadset[i].busy ){ 505 505 Tcl_AppendResult(interp, "no such thread", 0); 506 506 return TCL_ERROR; 507 507 } 508 - thread_wait(&threadset[i]); 508 + test_thread_wait(&threadset[i]); 509 509 threadset[i].xOp = do_step; 510 510 threadset[i].opnum++; 511 511 return TCL_OK; 512 512 } 513 513 514 514 /* 515 515 ** This procedure runs in the thread to finalize a virtual machine. ................................................................................ 543 543 } 544 544 i = parse_thread_id(interp, argv[1]); 545 545 if( i<0 ) return TCL_ERROR; 546 546 if( !threadset[i].busy ){ 547 547 Tcl_AppendResult(interp, "no such thread", 0); 548 548 return TCL_ERROR; 549 549 } 550 - thread_wait(&threadset[i]); 550 + test_thread_wait(&threadset[i]); 551 551 threadset[i].xOp = do_finalize; 552 552 sqlite3_free(threadset[i].zArg); 553 553 threadset[i].zArg = 0; 554 554 threadset[i].opnum++; 555 555 return TCL_OK; 556 556 } 557 557 ................................................................................ 575 575 } 576 576 i = parse_thread_id(interp, argv[1]); 577 577 if( i<0 ) return TCL_ERROR; 578 578 if( !threadset[i].busy ){ 579 579 Tcl_AppendResult(interp, "no such thread", 0); 580 580 return TCL_ERROR; 581 581 } 582 - thread_wait(&threadset[i]); 582 + test_thread_wait(&threadset[i]); 583 583 j = parse_thread_id(interp, argv[2]); 584 584 if( j<0 ) return TCL_ERROR; 585 585 if( !threadset[j].busy ){ 586 586 Tcl_AppendResult(interp, "no such thread", 0); 587 587 return TCL_ERROR; 588 588 } 589 - thread_wait(&threadset[j]); 589 + test_thread_wait(&threadset[j]); 590 590 temp = threadset[i].db; 591 591 threadset[i].db = threadset[j].db; 592 592 threadset[j].db = temp; 593 593 return TCL_OK; 594 594 } 595 595 596 596 /* ................................................................................ 616 616 } 617 617 i = parse_thread_id(interp, argv[1]); 618 618 if( i<0 ) return TCL_ERROR; 619 619 if( !threadset[i].busy ){ 620 620 Tcl_AppendResult(interp, "no such thread", 0); 621 621 return TCL_ERROR; 622 622 } 623 - thread_wait(&threadset[i]); 623 + test_thread_wait(&threadset[i]); 624 624 sqlite3TestMakePointerStr(interp, zBuf, threadset[i].db); 625 625 threadset[i].db = 0; 626 626 Tcl_AppendResult(interp, zBuf, (char*)0); 627 627 return TCL_OK; 628 628 } 629 629 630 630 /* ................................................................................ 647 647 } 648 648 i = parse_thread_id(interp, argv[1]); 649 649 if( i<0 ) return TCL_ERROR; 650 650 if( !threadset[i].busy ){ 651 651 Tcl_AppendResult(interp, "no such thread", 0); 652 652 return TCL_ERROR; 653 653 } 654 - thread_wait(&threadset[i]); 654 + test_thread_wait(&threadset[i]); 655 655 assert( !threadset[i].db ); 656 656 threadset[i].db = (sqlite3*)sqlite3TestTextToPtr(argv[2]); 657 657 return TCL_OK; 658 658 } 659 659 660 660 /* 661 661 ** Usage: thread_stmt_get ID ................................................................................ 679 679 } 680 680 i = parse_thread_id(interp, argv[1]); 681 681 if( i<0 ) return TCL_ERROR; 682 682 if( !threadset[i].busy ){ 683 683 Tcl_AppendResult(interp, "no such thread", 0); 684 684 return TCL_ERROR; 685 685 } 686 - thread_wait(&threadset[i]); 686 + test_thread_wait(&threadset[i]); 687 687 sqlite3TestMakePointerStr(interp, zBuf, threadset[i].pStmt); 688 688 threadset[i].pStmt = 0; 689 689 Tcl_AppendResult(interp, zBuf, (char*)0); 690 690 return TCL_OK; 691 691 } 692 692 693 693 /*
Changes to src/test_fs.c.
736 736 737 737 int n; 738 738 fd = open(zFile, O_RDONLY); 739 739 if( fd<0 ) return SQLITE_IOERR; 740 740 fstat(fd, &sbuf); 741 741 742 742 if( sbuf.st_size>=pCur->nAlloc ){ 743 - int nNew = sbuf.st_size*2; 743 + sqlite3_int64 nNew = sbuf.st_size*2; 744 744 char *zNew; 745 745 if( nNew<1024 ) nNew = 1024; 746 746 747 747 zNew = sqlite3Realloc(pCur->zBuf, nNew); 748 748 if( zNew==0 ){ 749 749 close(fd); 750 750 return SQLITE_NOMEM;
Changes to src/test_vfs.c.
231 231 { SQLITE_OK, "SQLITE_OK" }, 232 232 { SQLITE_ERROR, "SQLITE_ERROR" }, 233 233 { SQLITE_IOERR, "SQLITE_IOERR" }, 234 234 { SQLITE_LOCKED, "SQLITE_LOCKED" }, 235 235 { SQLITE_BUSY, "SQLITE_BUSY" }, 236 236 { SQLITE_READONLY, "SQLITE_READONLY" }, 237 237 { SQLITE_READONLY_CANTINIT, "SQLITE_READONLY_CANTINIT" }, 238 + { -1, "SQLITE_OMIT" }, 238 239 }; 239 240 240 241 const char *z; 241 242 int i; 242 243 243 244 z = Tcl_GetStringResult(p->interp); 244 245 for(i=0; i<ArraySize(aCode); i++){ ................................................................................ 378 379 379 380 if( p->pScript && p->mask&TESTVFS_WRITE_MASK ){ 380 381 tvfsExecTcl(p, "xWrite", 381 382 Tcl_NewStringObj(pFd->zFilename, -1), pFd->pShmId, 382 383 Tcl_NewWideIntObj(iOfst), Tcl_NewIntObj(iAmt) 383 384 ); 384 385 tvfsResultCode(p, &rc); 386 + if( rc<0 ) return SQLITE_OK; 385 387 } 386 388 387 389 if( rc==SQLITE_OK && tvfsInjectFullerr(p) ){ 388 390 rc = SQLITE_FULL; 389 391 } 390 392 if( rc==SQLITE_OK && p->mask&TESTVFS_WRITE_MASK && tvfsInjectIoerr(p) ){ 391 393 rc = SQLITE_IOERR;
Changes to src/utf.c.
196 196 #ifndef SQLITE_OMIT_UTF16 197 197 /* 198 198 ** This routine transforms the internal text encoding used by pMem to 199 199 ** desiredEnc. It is an error if the string is already of the desired 200 200 ** encoding, or if *pMem does not contain a string value. 201 201 */ 202 202 SQLITE_NOINLINE int sqlite3VdbeMemTranslate(Mem *pMem, u8 desiredEnc){ 203 - int len; /* Maximum length of output string in bytes */ 204 - unsigned char *zOut; /* Output buffer */ 205 - unsigned char *zIn; /* Input iterator */ 206 - unsigned char *zTerm; /* End of input */ 207 - unsigned char *z; /* Output iterator */ 203 + sqlite3_int64 len; /* Maximum length of output string in bytes */ 204 + unsigned char *zOut; /* Output buffer */ 205 + unsigned char *zIn; /* Input iterator */ 206 + unsigned char *zTerm; /* End of input */ 207 + unsigned char *z; /* Output iterator */ 208 208 unsigned int c; 209 209 210 210 assert( pMem->db==0 || sqlite3_mutex_held(pMem->db->mutex) ); 211 211 assert( pMem->flags&MEM_Str ); 212 212 assert( pMem->enc!=desiredEnc ); 213 213 assert( pMem->enc!=0 ); 214 214 assert( pMem->n>=0 ); ................................................................................ 249 249 if( desiredEnc==SQLITE_UTF8 ){ 250 250 /* When converting from UTF-16, the maximum growth results from 251 251 ** translating a 2-byte character to a 4-byte UTF-8 character. 252 252 ** A single byte is required for the output string 253 253 ** nul-terminator. 254 254 */ 255 255 pMem->n &= ~1; 256 - len = pMem->n * 2 + 1; 256 + len = 2 * (sqlite3_int64)pMem->n + 1; 257 257 }else{ 258 258 /* When converting from UTF-8 to UTF-16 the maximum growth is caused 259 259 ** when a 1-byte UTF-8 character is translated into a 2-byte UTF-16 260 260 ** character. Two bytes are required in the output buffer for the 261 261 ** nul-terminator. 262 262 */ 263 - len = pMem->n * 2 + 2; 263 + len = 2 * (sqlite3_int64)pMem->n + 2; 264 264 } 265 265 266 266 /* Set zIn to point at the start of the input buffer and zTerm to point 1 267 267 ** byte past the end. 268 268 ** 269 269 ** Variable zOut is set to point at the output buffer, space obtained 270 270 ** from sqlite3_malloc().
Changes to src/util.c.
28 28 void sqlite3Coverage(int x){ 29 29 static unsigned dummy = 0; 30 30 dummy += (unsigned)x; 31 31 } 32 32 #endif 33 33 34 34 /* 35 -** Give a callback to the test harness that can be used to simulate faults 36 -** in places where it is difficult or expensive to do so purely by means 37 -** of inputs. 35 +** Calls to sqlite3FaultSim() are used to simulate a failure during testing, 36 +** or to bypass normal error detection during testing in order to let 37 +** execute proceed futher downstream. 38 38 ** 39 -** The intent of the integer argument is to let the fault simulator know 40 -** which of multiple sqlite3FaultSim() calls has been hit. 39 +** In deployment, sqlite3FaultSim() *always* return SQLITE_OK (0). The 40 +** sqlite3FaultSim() function only returns non-zero during testing. 41 41 ** 42 -** Return whatever integer value the test callback returns, or return 43 -** SQLITE_OK if no test callback is installed. 42 +** During testing, if the test harness has set a fault-sim callback using 43 +** a call to sqlite3_test_control(SQLITE_TESTCTRL_FAULT_INSTALL), then 44 +** each call to sqlite3FaultSim() is relayed to that application-supplied 45 +** callback and the integer return value form the application-supplied 46 +** callback is returned by sqlite3FaultSim(). 47 +** 48 +** The integer argument to sqlite3FaultSim() is a code to identify which 49 +** sqlite3FaultSim() instance is being invoked. Each call to sqlite3FaultSim() 50 +** should have a unique code. To prevent legacy testing applications from 51 +** breaking, the codes should not be changed or reused. 44 52 */ 45 53 #ifndef SQLITE_UNTESTABLE 46 54 int sqlite3FaultSim(int iTest){ 47 55 int (*xCallback)(int) = sqlite3GlobalConfig.xTestCallback; 48 56 return xCallback ? xCallback(iTest) : SQLITE_OK; 49 57 } 50 58 #endif ................................................................................ 220 228 }else{ 221 229 pParse->nErr++; 222 230 sqlite3DbFree(db, pParse->zErrMsg); 223 231 pParse->zErrMsg = zMsg; 224 232 pParse->rc = SQLITE_ERROR; 225 233 } 226 234 } 235 + 236 +/* 237 +** If database connection db is currently parsing SQL, then transfer 238 +** error code errCode to that parser if the parser has not already 239 +** encountered some other kind of error. 240 +*/ 241 +int sqlite3ErrorToParser(sqlite3 *db, int errCode){ 242 + Parse *pParse; 243 + if( db==0 || (pParse = db->pParse)==0 ) return errCode; 244 + pParse->rc = errCode; 245 + pParse->nErr++; 246 + return errCode; 247 +} 227 248 228 249 /* 229 250 ** Convert an SQL-style quoted string into a normal string by removing 230 251 ** the quote characters. The conversion is done in-place. If the 231 252 ** input does not begin with a quote character, then this routine 232 253 ** is a no-op. 233 254 ** ................................................................................ 1572 1593 char *z; /* Pointer to where zName will be stored */ 1573 1594 int i; /* Index in pIn[] where zName is stored */ 1574 1595 1575 1596 nInt = nName/4 + 3; 1576 1597 assert( pIn==0 || pIn[0]>=3 ); /* Verify ok to add new elements */ 1577 1598 if( pIn==0 || pIn[1]+nInt > pIn[0] ){ 1578 1599 /* Enlarge the allocation */ 1579 - int nAlloc = (pIn ? pIn[0]*2 : 10) + nInt; 1600 + sqlite3_int64 nAlloc = (pIn ? 2*(sqlite3_int64)pIn[0] : 10) + nInt; 1580 1601 VList *pOut = sqlite3DbRealloc(db, pIn, nAlloc*sizeof(int)); 1581 1602 if( pOut==0 ) return pIn; 1582 1603 if( pIn==0 ) pOut[1] = 2; 1583 1604 pIn = pOut; 1584 1605 pIn[0] = nAlloc; 1585 1606 } 1586 1607 i = pIn[1];
Changes to src/vdbeInt.h.
278 278 279 279 /* 280 280 ** Clear any existing type flags from a Mem and replace them with f 281 281 */ 282 282 #define MemSetTypeFlag(p, f) \ 283 283 ((p)->flags = ((p)->flags&~(MEM_TypeMask|MEM_Zero))|f) 284 284 285 +/* 286 +** True if Mem X is a NULL-nochng type. 287 +*/ 288 +#define MemNullNochng(X) \ 289 + ((X)->flags==(MEM_Null|MEM_Zero) && (X)->n==0 && (X)->u.nZero==0) 290 + 285 291 /* 286 292 ** Return true if a memory cell is not marked as invalid. This macro 287 293 ** is for use inside assert() statements only. 288 294 */ 289 295 #ifdef SQLITE_DEBUG 290 296 #define memIsValid(M) ((M)->flags & MEM_Undefined)==0 291 297 #endif
Changes to src/vdbeapi.c.
66 66 sqlite3_int64 iElapse; 67 67 assert( p->startTime>0 ); 68 68 assert( (db->mTrace & (SQLITE_TRACE_PROFILE|SQLITE_TRACE_XPROFILE))!=0 ); 69 69 assert( db->init.busy==0 ); 70 70 assert( p->zSql!=0 ); 71 71 sqlite3OsCurrentTimeInt64(db->pVfs, &iNow); 72 72 iElapse = (iNow - p->startTime)*1000000; 73 -#ifndef SQLITE_OMIT_DEPRECATED 73 +#ifndef SQLITE_OMIT_DEPRECATED 74 74 if( db->xProfile ){ 75 75 db->xProfile(db->pProfileArg, p->zSql, iElapse); 76 76 } 77 77 #endif 78 78 if( db->mTrace & SQLITE_TRACE_PROFILE ){ 79 79 db->xTrace(SQLITE_TRACE_PROFILE, db->pTraceArg, p, (void*)&iElapse); 80 80 }
Changes to src/vdbeaux.c.
151 151 ** more frequent reallocs and hence provide more opportunities for 152 152 ** simulated OOM faults. SQLITE_TEST_REALLOC_STRESS is generally used 153 153 ** during testing only. With SQLITE_TEST_REALLOC_STRESS grow the op array 154 154 ** by the minimum* amount required until the size reaches 512. Normal 155 155 ** operation (without SQLITE_TEST_REALLOC_STRESS) is to double the current 156 156 ** size of the op array or add 1KB of space, whichever is smaller. */ 157 157 #ifdef SQLITE_TEST_REALLOC_STRESS 158 - int nNew = (v->nOpAlloc>=512 ? v->nOpAlloc*2 : v->nOpAlloc+nOp); 158 + sqlite3_int64 nNew = (v->nOpAlloc>=512 ? 2*(sqlite3_int64)v->nOpAlloc 159 + : (sqlite3_int64)v->nOpAlloc+nOp); 159 160 #else 160 - int nNew = (v->nOpAlloc ? v->nOpAlloc*2 : (int)(1024/sizeof(Op))); 161 + sqlite3_int64 nNew = (v->nOpAlloc ? 2*(sqlite3_int64)v->nOpAlloc 162 + : (sqlite3_int64)(1024/sizeof(Op))); 161 163 UNUSED_PARAMETER(nOp); 162 164 #endif 163 165 164 166 /* Ensure that the size of a VDBE does not grow too large */ 165 167 if( nNew > p->db->aLimit[SQLITE_LIMIT_VDBE_OP] ){ 166 168 sqlite3OomFault(p->db); 167 169 return SQLITE_NOMEM; ................................................................................ 941 943 Vdbe *p, /* VM to add scanstatus() to */ 942 944 int addrExplain, /* Address of OP_Explain (or 0) */ 943 945 int addrLoop, /* Address of loop counter */ 944 946 int addrVisit, /* Address of rows visited counter */ 945 947 LogEst nEst, /* Estimated number of output rows */ 946 948 const char *zName /* Name of table or index being scanned */ 947 949 ){ 948 - int nByte = (p->nScan+1) * sizeof(ScanStatus); 950 + sqlite3_int64 nByte = (p->nScan+1) * sizeof(ScanStatus); 949 951 ScanStatus *aNew; 950 952 aNew = (ScanStatus*)sqlite3DbRealloc(p->db, p->aScan, nByte); 951 953 if( aNew ){ 952 954 ScanStatus *pNew = &aNew[p->nScan++]; 953 955 pNew->addrExplain = addrExplain; 954 956 pNew->addrLoop = addrLoop; 955 957 pNew->addrVisit = addrVisit; ................................................................................ 2062 2064 #endif /* !SQLITE_OMIT_TRACE && SQLITE_ENABLE_IOTRACE */ 2063 2065 2064 2066 /* An instance of this object describes bulk memory available for use 2065 2067 ** by subcomponents of a prepared statement. Space is allocated out 2066 2068 ** of a ReusableSpace object by the allocSpace() routine below. 2067 2069 */ 2068 2070 struct ReusableSpace { 2069 - u8 *pSpace; /* Available memory */ 2070 - int nFree; /* Bytes of available memory */ 2071 - int nNeeded; /* Total bytes that could not be allocated */ 2071 + u8 *pSpace; /* Available memory */ 2072 + sqlite3_int64 nFree; /* Bytes of available memory */ 2073 + sqlite3_int64 nNeeded; /* Total bytes that could not be allocated */ 2072 2074 }; 2073 2075 2074 2076 /* Try to allocate nByte bytes of 8-byte aligned bulk memory for pBuf 2075 2077 ** from the ReusableSpace object. Return a pointer to the allocated 2076 2078 ** memory on success. If insufficient memory is available in the 2077 2079 ** ReusableSpace object, increase the ReusableSpace.nNeeded 2078 2080 ** value by the amount needed and return NULL. ................................................................................ 2084 2086 ** This allocator is employed to repurpose unused slots at the end of the 2085 2087 ** opcode array of prepared state for other memory needs of the prepared 2086 2088 ** statement. 2087 2089 */ 2088 2090 static void *allocSpace( 2089 2091 struct ReusableSpace *p, /* Bulk memory available for allocation */ 2090 2092 void *pBuf, /* Pointer to a prior allocation */ 2091 - int nByte /* Bytes of memory needed */ 2093 + sqlite3_int64 nByte /* Bytes of memory needed */ 2092 2094 ){ 2093 2095 assert( EIGHT_BYTE_ALIGNMENT(p->pSpace) ); 2094 2096 if( pBuf==0 ){ 2095 2097 nByte = ROUND8(nByte); 2096 2098 if( nByte <= p->nFree ){ 2097 2099 p->nFree -= nByte; 2098 2100 pBuf = &p->pSpace[p->nFree];
Changes to src/vdbemem.c.
294 294 ** If the given Mem* has a zero-filled tail, turn it into an ordinary 295 295 ** blob stored in dynamically allocated space. 296 296 */ 297 297 #ifndef SQLITE_OMIT_INCRBLOB 298 298 int sqlite3VdbeMemExpandBlob(Mem *pMem){ 299 299 int nByte; 300 300 assert( pMem->flags & MEM_Zero ); 301 - assert( pMem->flags&MEM_Blob ); 301 + assert( (pMem->flags&MEM_Blob)!=0 || MemNullNochng(pMem) ); 302 + testcase( sqlite3_value_nochange(pMem) ); 302 303 assert( !sqlite3VdbeMemIsRowSet(pMem) ); 303 304 assert( pMem->db==0 || sqlite3_mutex_held(pMem->db->mutex) ); 304 305 305 306 /* Set nByte to the number of bytes required to store the expanded blob. */ 306 307 nByte = pMem->n + pMem->u.nZero; 307 308 if( nByte<=0 ){ 309 + if( (pMem->flags & MEM_Blob)==0 ) return SQLITE_OK; 308 310 nByte = 1; 309 311 } 310 312 if( sqlite3VdbeMemGrow(pMem, nByte, 1) ){ 311 313 return SQLITE_NOMEM_BKPT; 312 314 } 313 315 314 316 memset(&pMem->z[pMem->n], 0, pMem->u.nZero); ................................................................................ 1057 1059 */ 1058 1060 if( xDel==SQLITE_TRANSIENT ){ 1059 1061 u32 nAlloc = nByte; 1060 1062 if( flags&MEM_Term ){ 1061 1063 nAlloc += (enc==SQLITE_UTF8?1:2); 1062 1064 } 1063 1065 if( nByte>iLimit ){ 1064 - return SQLITE_TOOBIG; 1066 + return sqlite3ErrorToParser(pMem->db, SQLITE_TOOBIG); 1065 1067 } 1066 1068 testcase( nAlloc==0 ); 1067 1069 testcase( nAlloc==31 ); 1068 1070 testcase( nAlloc==32 ); 1069 1071 if( sqlite3VdbeMemClearAndResize(pMem, (int)MAX(nAlloc,32)) ){ 1070 1072 return SQLITE_NOMEM_BKPT; 1071 1073 }
Changes to src/vdbesort.c.
533 533 ** In this case, allocate space at p->aAlloc[] to copy the requested 534 534 ** range into. Then return a copy of pointer p->aAlloc to the caller. */ 535 535 int nRem; /* Bytes remaining to copy */ 536 536 537 537 /* Extend the p->aAlloc[] allocation if required. */ 538 538 if( p->nAlloc<nByte ){ 539 539 u8 *aNew; 540 - int nNew = MAX(128, p->nAlloc*2); 540 + sqlite3_int64 nNew = MAX(128, 2*(sqlite3_int64)p->nAlloc); 541 541 while( nByte>nNew ) nNew = nNew*2; 542 542 aNew = sqlite3Realloc(p->aAlloc, nNew); 543 543 if( !aNew ) return SQLITE_NOMEM_BKPT; 544 544 p->nAlloc = nNew; 545 545 p->aAlloc = aNew; 546 546 } 547 547 ................................................................................ 1825 1825 1826 1826 if( pSorter->list.aMemory ){ 1827 1827 int nMin = pSorter->iMemory + nReq; 1828 1828 1829 1829 if( nMin>pSorter->nMemory ){ 1830 1830 u8 *aNew; 1831 1831 int iListOff = (u8*)pSorter->list.pList - pSorter->list.aMemory; 1832 - int nNew = pSorter->nMemory * 2; 1832 + sqlite3_int64 nNew = 2 * (sqlite3_int64)pSorter->nMemory; 1833 1833 while( nNew < nMin ) nNew = nNew*2; 1834 1834 if( nNew > pSorter->mxPmaSize ) nNew = pSorter->mxPmaSize; 1835 1835 if( nNew < nMin ) nNew = nMin; 1836 1836 1837 1837 aNew = sqlite3Realloc(pSorter->list.aMemory, nNew); 1838 1838 if( !aNew ) return SQLITE_NOMEM_BKPT; 1839 1839 pSorter->list.pList = (SorterRecord*)&aNew[iListOff];
Changes to src/vtab.c.
298 298 299 299 /* 300 300 ** Add a new module argument to pTable->azModuleArg[]. 301 301 ** The string is not copied - the pointer is stored. The 302 302 ** string will be freed automatically when the table is 303 303 ** deleted. 304 304 */ 305 -static void addModuleArgument(sqlite3 *db, Table *pTable, char *zArg){ 306 - int nBytes = sizeof(char *)*(2+pTable->nModuleArg); 305 +static void addModuleArgument(Parse *pParse, Table *pTable, char *zArg){ 306 + sqlite3_int64 nBytes = sizeof(char *)*(2+pTable->nModuleArg); 307 307 char **azModuleArg; 308 + sqlite3 *db = pParse->db; 309 + if( pTable->nModuleArg+3>=db->aLimit[SQLITE_LIMIT_COLUMN] ){ 310 + sqlite3ErrorMsg(pParse, "too many columns on %s", pTable->zName); 311 + } 308 312 azModuleArg = sqlite3DbRealloc(db, pTable->azModuleArg, nBytes); 309 313 if( azModuleArg==0 ){ 310 314 sqlite3DbFree(db, zArg); 311 315 }else{ 312 316 int i = pTable->nModuleArg++; 313 317 azModuleArg[i] = zArg; 314 318 azModuleArg[i+1] = 0; ................................................................................ 335 339 pTable = pParse->pNewTable; 336 340 if( pTable==0 ) return; 337 341 assert( 0==pTable->pIndex ); 338 342 339 343 db = pParse->db; 340 344 341 345 assert( pTable->nModuleArg==0 ); 342 - addModuleArgument(db, pTable, sqlite3NameFromToken(db, pModuleName)); 343 - addModuleArgument(db, pTable, 0); 344 - addModuleArgument(db, pTable, sqlite3DbStrDup(db, pTable->zName)); 346 + addModuleArgument(pParse, pTable, sqlite3NameFromToken(db, pModuleName)); 347 + addModuleArgument(pParse, pTable, 0); 348 + addModuleArgument(pParse, pTable, sqlite3DbStrDup(db, pTable->zName)); 345 349 assert( (pParse->sNameToken.z==pName2->z && pName2->z!=0) 346 350 || (pParse->sNameToken.z==pName1->z && pName2->z==0) 347 351 ); 348 352 pParse->sNameToken.n = (int)( 349 353 &pModuleName->z[pModuleName->n] - pParse->sNameToken.z 350 354 ); 351 355 ................................................................................ 370 374 ** virtual table currently under construction in pParse->pTable. 371 375 */ 372 376 static void addArgumentToVtab(Parse *pParse){ 373 377 if( pParse->sArg.z && pParse->pNewTable ){ 374 378 const char *z = (const char*)pParse->sArg.z; 375 379 int n = pParse->sArg.n; 376 380 sqlite3 *db = pParse->db; 377 - addModuleArgument(db, pParse->pNewTable, sqlite3DbStrNDup(db, z, n)); 381 + addModuleArgument(pParse, pParse->pNewTable, sqlite3DbStrNDup(db, z, n)); 378 382 } 379 383 } 380 384 381 385 /* 382 386 ** The parser calls this routine after the CREATE VIRTUAL TABLE statement 383 387 ** has been completely parsed. 384 388 */ ................................................................................ 659 663 */ 660 664 static int growVTrans(sqlite3 *db){ 661 665 const int ARRAY_INCR = 5; 662 666 663 667 /* Grow the sqlite3.aVTrans array if required */ 664 668 if( (db->nVTrans%ARRAY_INCR)==0 ){ 665 669 VTable **aVTrans; 666 - int nBytes = sizeof(sqlite3_vtab *) * (db->nVTrans + ARRAY_INCR); 670 + sqlite3_int64 nBytes = sizeof(sqlite3_vtab*)* 671 + ((sqlite3_int64)db->nVTrans + ARRAY_INCR); 667 672 aVTrans = sqlite3DbRealloc(db, (void *)db->aVTrans, nBytes); 668 673 if( !aVTrans ){ 669 674 return SQLITE_NOMEM_BKPT; 670 675 } 671 676 memset(&aVTrans[db->nVTrans], 0, sizeof(sqlite3_vtab *)*ARRAY_INCR); 672 677 db->aVTrans = aVTrans; 673 678 } ................................................................................ 1155 1160 return 0; 1156 1161 } 1157 1162 pMod->pEpoTab = pTab; 1158 1163 pTab->nTabRef = 1; 1159 1164 pTab->pSchema = db->aDb[0].pSchema; 1160 1165 assert( pTab->nModuleArg==0 ); 1161 1166 pTab->iPKey = -1; 1162 - addModuleArgument(db, pTab, sqlite3DbStrDup(db, pTab->zName)); 1163 - addModuleArgument(db, pTab, 0); 1164 - addModuleArgument(db, pTab, sqlite3DbStrDup(db, pTab->zName)); 1167 + addModuleArgument(pParse, pTab, sqlite3DbStrDup(db, pTab->zName)); 1168 + addModuleArgument(pParse, pTab, 0); 1169 + addModuleArgument(pParse, pTab, sqlite3DbStrDup(db, pTab->zName)); 1165 1170 rc = vtabCallConstructor(db, pTab, pMod, pModule->xConnect, &zErr); 1166 1171 if( rc ){ 1167 1172 sqlite3ErrorMsg(pParse, "%s", zErr); 1168 1173 sqlite3DbFree(db, zErr); 1169 1174 sqlite3VtabEponymousTableClear(db, pMod); 1170 1175 return 0; 1171 1176 }
Changes to src/wal.c.
571 571 int iPage, /* The page we seek */ 572 572 volatile u32 **ppPage /* Write the page pointer here */ 573 573 ){ 574 574 int rc = SQLITE_OK; 575 575 576 576 /* Enlarge the pWal->apWiData[] array if required */ 577 577 if( pWal->nWiData<=iPage ){ 578 - int nByte = sizeof(u32*)*(iPage+1); 578 + sqlite3_int64 nByte = sizeof(u32*)*(iPage+1); 579 579 volatile u32 **apNew; 580 580 apNew = (volatile u32 **)sqlite3_realloc64((void *)pWal->apWiData, nByte); 581 581 if( !apNew ){ 582 582 *ppPage = 0; 583 583 return SQLITE_NOMEM_BKPT; 584 584 } 585 585 memset((void*)&apNew[pWal->nWiData], 0, ................................................................................ 675 675 s2 = aIn[1]; 676 676 }else{ 677 677 s1 = s2 = 0; 678 678 } 679 679 680 680 assert( nByte>=8 ); 681 681 assert( (nByte&0x00000007)==0 ); 682 + assert( nByte<=65536 ); 682 683 683 684 if( nativeCksum ){ 684 685 do { 685 686 s1 += *aData++ + s2; 686 687 s2 += *aData++ + s1; 687 688 }while( aData<aEnd ); 688 689 }else{ ................................................................................ 1617 1618 ** The calling routine should invoke walIteratorFree() to destroy the 1618 1619 ** WalIterator object when it has finished with it. 1619 1620 */ 1620 1621 static int walIteratorInit(Wal *pWal, u32 nBackfill, WalIterator **pp){ 1621 1622 WalIterator *p; /* Return value */ 1622 1623 int nSegment; /* Number of segments to merge */ 1623 1624 u32 iLast; /* Last frame in log */ 1624 - int nByte; /* Number of bytes to allocate */ 1625 + sqlite3_int64 nByte; /* Number of bytes to allocate */ 1625 1626 int i; /* Iterator variable */ 1626 1627 ht_slot *aTmp; /* Temp space used by merge-sort */ 1627 1628 int rc = SQLITE_OK; /* Return Code */ 1628 1629 1629 1630 /* This routine only runs while holding the checkpoint lock. And 1630 1631 ** it only runs if there is actually content in the log (mxFrame>0). 1631 1632 */
Changes to src/wherecode.c.
1963 1963 if( (pWC->a[iTerm].wtFlags & (TERM_VIRTUAL|TERM_CODED))!=0 ) continue; 1964 1964 if( (pWC->a[iTerm].eOperator & WO_ALL)==0 ) continue; 1965 1965 testcase( pWC->a[iTerm].wtFlags & TERM_ORINFO ); 1966 1966 pExpr = sqlite3ExprDup(db, pExpr, 0); 1967 1967 pAndExpr = sqlite3ExprAnd(db, pAndExpr, pExpr); 1968 1968 } 1969 1969 if( pAndExpr ){ 1970 - pAndExpr = sqlite3PExpr(pParse, TK_AND|TKFLG_DONTFOLD, 0, pAndExpr); 1970 + /* The extra 0x10000 bit on the opcode is masked off and does not 1971 + ** become part of the new Expr.op. However, it does make the 1972 + ** op==TK_AND comparison inside of sqlite3PExpr() false, and this 1973 + ** prevents sqlite3PExpr() from implementing AND short-circuit 1974 + ** optimization, which we do not want here. */ 1975 + pAndExpr = sqlite3PExpr(pParse, TK_AND|0x10000, 0, pAndExpr); 1971 1976 } 1972 1977 } 1973 1978 1974 1979 /* Run a separate WHERE clause for each term of the OR clause. After 1975 1980 ** eliminating duplicates from other WHERE clauses, the action for each 1976 1981 ** sub-WHERE clause is to to invoke the main loop body as a subroutine. 1977 1982 */
Changes to test/altertab3.test.
95 95 BEGIN; 96 96 ALTER TABLE t3 RENAME TO t4; 97 97 } {1 {error in trigger tr1: no such table: main.t2}} 98 98 do_execsql_test 4.1.2 { 99 99 COMMIT; 100 100 } 101 101 do_execsql_test 4.1.3 { 102 - SELECT * FROM sqlite_master WHERE type='table' AND name!='t1'; 103 -} {table t3 t3 3 {CREATE TABLE t3(e, f)}} 102 + SELECT type, name, tbl_name, sql 103 + FROM sqlite_master WHERE type='table' AND name!='t1'; 104 +} {table t3 t3 {CREATE TABLE t3(e, f)}} 104 105 105 106 106 107 do_catchsql_test 4.2.1 { 107 108 BEGIN; 108 109 ALTER TABLE t3 RENAME e TO eee; 109 110 } {1 {error in trigger tr1: no such table: main.t2}} 110 111 do_execsql_test 4.2.2 { 111 112 COMMIT; 112 113 } 113 114 do_execsql_test 4.2.3 { 114 - SELECT * FROM sqlite_master WHERE type='table' AND name!='t1'; 115 -} {table t3 t3 3 {CREATE TABLE t3(e, f)}} 115 + SELECT type, name, tbl_name, sql 116 + FROM sqlite_master WHERE type='table' AND name!='t1'; 117 +} {table t3 t3 {CREATE TABLE t3(e, f)}} 116 118 117 119 #------------------------------------------------------------------------- 118 120 reset_db 119 121 do_execsql_test 5.0 { 120 122 CREATE TABLE t1 ( 121 123 c1 integer, c2, PRIMARY KEY(c1 collate rtrim), 122 124 UNIQUE(c2)
Changes to test/badutf2.test.
94 94 do_test badutf2-3.1.$i { 95 95 set sql "SELECT hex('$hstr') AS x;" 96 96 set res [ sqlite3_exec db $sql ] 97 97 lindex [ lindex $res 1] 1 98 98 } $uval 99 99 } 100 100 101 - do_test badutf2-4.1.$i { 102 - sqlite3_reset $S 103 - sqlite3_bind_text $S 1 $xstr $len 104 - sqlite3_step $S 105 - utf8_to_ustr2 [ sqlite3_column_text $S 0 ] 106 - } $ustr 101 + # Tcl 8.7 and later do automatic bad-utf8 correction for 102 + # characters 0x80 thru 0x9f so test case 5 does not work here. 103 + if {$i==5 && $tcl_version>=8.7} { 104 + # no-op 105 + } else { 106 + do_test badutf2-4.1.$i { 107 + sqlite3_reset $S 108 + sqlite3_bind_text $S 1 $xstr $len 109 + sqlite3_step $S 110 + utf8_to_ustr2 [ sqlite3_column_text $S 0 ] 111 + } $ustr 112 + } 107 113 108 114 ifcapable debug { 109 115 do_test badutf2-5.1.$i { 110 116 utf8_to_utf8 $uval 111 117 } $u2u 112 118 } 113 119
Changes to test/corruptC.test.
93 93 forcecopy test.bu test.db 94 94 95 95 # insert corrupt byte(s) 96 96 hexio_write test.db 2053 [format %02x 0x04] 97 97 98 98 sqlite3 db test.db 99 99 catchsql {PRAGMA integrity_check} 100 -} {1 {database disk image is malformed}} 100 +} {0 {{*** in database main *** 101 +Page 3: free space corruption}}} 101 102 102 103 # test that a corrupt content offset size is handled (seed 5649) 103 104 # 104 105 # Update 2016-12-27: As of check-in [0b86fbca66] "In sqlite3BtreeInsert() when 105 106 # replacing a re-existing row, try to overwrite the cell directly rather than 106 107 # deallocate and reallocate the cell" on 2016-12-09, this test case no longer 107 108 # detects the offset size problem during the UPDATE. We have to run a subsequent
Changes to test/dbfuzz001.test.
185 185 186 186 # The DELETE query below deletes the very last cell from page 8. 187 187 # Prior to a certain fix to sqlite3BtreeDelete() and because of the 188 188 # corruption to the freeblock list on page 8, this would fail to 189 189 # cause a rebalance operation, which would leave the btree in a weird 190 190 # state that would lead to segfaults and or assertion faults. 191 191 # 192 -set res {0 {}} 193 -ifcapable oversize_cell_check { set res {1 {database disk image is malformed}} } 194 -do_catchsql_test dbfuzz001-110 { 192 +do_execsql_test dbfuzz001-110 { 195 193 DELETE FROM t3 WHERE x IS NOT NULL AND +rowid=6; 196 -} $res 194 +} {} 197 195 198 196 # This is a dbfuzz2-generate test case that can cause a page with 199 197 # pPage->nCell==0 to enter the balancer. 200 198 # 201 199 do_test dbfuzz001-200 { 202 200 db deserialize [decode_hexdb { 203 201 | size 3076 pagesize 512 filename c03.db
Changes to test/fts3varint.test.
106 106 576460752303423487 576460752303423488 576460752303423489 } 107 107 do_fts3_varint_test 2.60 { 108 108 1152921504606846975 1152921504606846976 1152921504606846977 } 109 109 do_fts3_varint_test 2.61 { 110 110 2305843009213693951 2305843009213693952 2305843009213693953 } 111 111 do_fts3_varint_test 2.62 { 112 112 4611686018427387903 4611686018427387904 4611686018427387905 } 113 -do_fts3_varint_test 2.63 { 114 - 9223372036854775807 9223372036854775808 9223372036854775809 } 113 + 114 +if {![catch {fts3_test_varint 18446744073709551615}]} { 115 + do_fts3_varint_test 2.63 { 116 + 9223372036854775807 9223372036854775808 9223372036854775809 } 115 117 116 -do_fts3_varint_test 3.0 { 18446744073709551615 -18446744073709551615 } 118 + do_fts3_varint_test 3.0 { 18446744073709551615 -18446744073709551615 } 119 +} 117 120 118 121 finish_test
Changes to test/fuzzdata8.db.
cannot compute difference between binary files
Changes to test/journal3.test.
34 34 1 00644 35 35 2 00666 36 36 3 00600 37 37 4 00755 38 38 } { 39 39 db close 40 40 #set effective [format %.5o [expr $permissions & ~$umask]] 41 + if {$tcl_version>=8.7} { 42 + regsub {^00} $permissions {0o} permissions 43 + } 41 44 set effective $permissions 42 45 do_test journal3-1.2.$tn.1 { 43 46 catch { forcedelete test.db-journal } 44 47 file attributes test.db -permissions $permissions 45 48 file attributes test.db -permissions 46 49 } $permissions 47 50 do_test journal3-1.2.$tn.2 { file exists test.db-journal } {0}
Changes to test/memdb1.test.
182 182 } {1 {unknown option: a}} 183 183 do_test 620 { 184 184 set rc [catch {db serialize a b} msg] 185 185 lappend rc $msg 186 186 } {1 {wrong # args: should be "db serialize ?DATABASE?"}} 187 187 188 188 #------------------------------------------------------------------------- 189 -reset_db 190 -do_execsql_test 700 { 191 - CREATE TABLE t1(a, b); 192 - PRAGMA schema_version = 0; 193 -} 194 -do_test 710 { 195 - set ser [db serialize main] 196 - db close 197 - sqlite3 db 198 - db deserialize main $ser 199 - catchsql { 200 - CREATE VIRTUAL TABLE t1 USING rtree(id, a, b, c, d); 189 +ifcapable vtab { 190 + reset_db 191 + do_execsql_test 700 { 192 + CREATE TABLE t1(a, b); 193 + PRAGMA schema_version = 0; 201 194 } 202 -} {1 {table t1 already exists}} 195 + do_test 710 { 196 + set ser [db serialize main] 197 + db close 198 + sqlite3 db 199 + db deserialize main $ser 200 + catchsql { 201 + CREATE VIRTUAL TABLE t1 USING rtree(id, a, b, c, d); 202 + } 203 + } {1 {table t1 already exists}} 204 +} 203 205 204 206 finish_test
Changes to test/pragma4.test.
172 172 } 173 173 do_test 4.3.4 { 174 174 sqlite3 db3 test.db 175 175 sqlite3 db2 test.db2 176 176 execsql { DROP INDEX i1 } db3 177 177 execsql { DROP INDEX i2 } db2 178 178 } {} 179 +if {[permutation]=="prepare"} { catchsql { SELECT * FROM sqlite_master } } 179 180 ifcapable vtab { 180 181 do_execsql_test 4.3.5 { SELECT * FROM pragma_index_info('i1') } 181 182 do_execsql_test 4.3.6 { SELECT * FROM pragma_index_info('i2') } 182 183 } 183 184 184 185 execsql {SELECT * FROM main.sqlite_master, aux.sqlite_master} 185 186 do_execsql_test 4.4.0 { ................................................................................ 190 191 do_execsql_test 4.4.1 { SELECT * FROM pragma_index_list('t1') } {0 i1 0 c 0} 191 192 do_execsql_test 4.4.2 { SELECT * FROM pragma_index_list('t2') } {0 i2 0 c 0} 192 193 } 193 194 do_test 4.4.3 { 194 195 execsql { DROP INDEX i1 } db3 195 196 execsql { DROP INDEX i2 } db2 196 197 } {} 198 +if {[permutation]=="prepare"} { 199 + catchsql { SELECT * FROM sqlite_master, aux.sqlite_master } 200 +} 197 201 ifcapable vtab { 198 202 do_execsql_test 4.4.5 { SELECT * FROM pragma_index_list('t1') } {} 199 203 do_execsql_test 4.4.6 { SELECT * FROM pragma_index_list('t2') } {} 200 204 } 201 205 execsql {SELECT * FROM main.sqlite_master, aux.sqlite_master} 202 206 203 207 do_execsql_test 4.5.0 { ................................................................................ 214 218 0 0 t2 r d {NO ACTION} {NO ACTION} NONE 215 219 } 216 220 } 217 221 do_test 4.5.3 { 218 222 execsql { DROP TABLE c1 } db3 219 223 execsql { DROP TABLE c2 } db2 220 224 } {} 225 +if {[permutation]=="prepare"} { 226 + catchsql { SELECT * FROM sqlite_master, aux.sqlite_master } 227 +} 221 228 ifcapable vtab { 222 229 do_execsql_test 4.5.4 { SELECT * FROM pragma_foreign_key_list('c1') } 223 230 do_execsql_test 4.5.5 { SELECT * FROM pragma_foreign_key_list('c2') } 224 231 } 225 232 execsql {SELECT * FROM main.sqlite_master, aux.sqlite_master} 226 233 227 234 do_execsql_test 4.6.0 {
Added test/releasetest_data.tcl.
1 + 2 +# This file contains Configuration data used by "wapptest.tcl" and 3 +# "releasetest.tcl". 4 +# 5 + 6 +# Omit comments (text between # and \n) in a long multi-line string. 7 +# 8 +proc strip_comments {in} { 9 + regsub -all {#[^\n]*\n} $in {} out 10 + return $out 11 +} 12 + 13 +array set ::Configs [strip_comments { 14 + "Default" { 15 + -O2 16 + --disable-amalgamation --disable-shared 17 + --enable-session 18 + -DSQLITE_ENABLE_DESERIALIZE 19 + } 20 + "Sanitize" { 21 + CC=clang -fsanitize=undefined 22 + -DSQLITE_ENABLE_STAT4 23 + --enable-session 24 + } 25 + "Stdcall" { 26 + -DUSE_STDCALL=1 27 + -O2 28 + } 29 + "Have-Not" { 30 + # The "Have-Not" configuration sets all possible -UHAVE_feature options 31 + # in order to verify that the code works even on platforms that lack 32 + # these support services. 33 + -DHAVE_FDATASYNC=0 34 + -DHAVE_GMTIME_R=0 35 + -DHAVE_ISNAN=0 36 + -DHAVE_LOCALTIME_R=0 37 + -DHAVE_LOCALTIME_S=0 38 + -DHAVE_MALLOC_USABLE_SIZE=0 39 + -DHAVE_STRCHRNUL=0 40 + -DHAVE_USLEEP=0 41 + -DHAVE_UTIME=0 42 + } 43 + "Unlock-Notify" { 44 + -O2 45 + -DSQLITE_ENABLE_UNLOCK_NOTIFY 46 + -DSQLITE_THREADSAFE 47 + -DSQLITE_TCL_DEFAULT_FULLMUTEX=1 48 + } 49 + "User-Auth" { 50 + -O2 51 + -DSQLITE_USER_AUTHENTICATION=1 52 + } 53 + "Secure-Delete" { 54 + -O2 55 + -DSQLITE_SECURE_DELETE=1 56 + -DSQLITE_SOUNDEX=1 57 + } 58 + "Update-Delete-Limit" { 59 + -O2 60 + -DSQLITE_DEFAULT_FILE_FORMAT=4 61 + -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1 62 + -DSQLITE_ENABLE_STMT_SCANSTATUS 63 + -DSQLITE_LIKE_DOESNT_MATCH_BLOBS 64 + -DSQLITE_ENABLE_CURSOR_HINTS 65 + --enable-json1 66 + } 67 + "Check-Symbols" { 68 + -DSQLITE_MEMDEBUG=1 69 + -DSQLITE_ENABLE_FTS3_PARENTHESIS=1 70 + -DSQLITE_ENABLE_FTS3=1 71 + -DSQLITE_ENABLE_RTREE=1 72 + -DSQLITE_ENABLE_MEMSYS5=1 73 + -DSQLITE_ENABLE_MEMSYS3=1 74 + -DSQLITE_ENABLE_COLUMN_METADATA=1 75 + -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1 76 + -DSQLITE_SECURE_DELETE=1 77 + -DSQLITE_SOUNDEX=1 78 + -DSQLITE_ENABLE_ATOMIC_WRITE=1 79 + -DSQLITE_ENABLE_MEMORY_MANAGEMENT=1 80 + -DSQLITE_ENABLE_OVERSIZE_CELL_CHECK=1 81 + -DSQLITE_ENABLE_STAT4 82 + -DSQLITE_ENABLE_STMT_SCANSTATUS 83 + --enable-json1 --enable-fts5 --enable-session 84 + } 85 + "Debug-One" { 86 + --disable-shared 87 + -O2 -funsigned-char 88 + -DSQLITE_DEBUG=1 89 + -DSQLITE_MEMDEBUG=1 90 + -DSQLITE_MUTEX_NOOP=1 91 + -DSQLITE_TCL_DEFAULT_FULLMUTEX=1 92 + -DSQLITE_ENABLE_FTS3=1 93 + -DSQLITE_ENABLE_RTREE=1 94 + -DSQLITE_ENABLE_MEMSYS5=1 95 + -DSQLITE_ENABLE_COLUMN_METADATA=1 96 + -DSQLITE_ENABLE_STAT4 97 + -DSQLITE_ENABLE_HIDDEN_COLUMNS 98 + -DSQLITE_MAX_ATTACHED=125 99 + -DSQLITE_MUTATION_TEST 100 + --enable-fts5 --enable-json1 101 + } 102 + "Fast-One" { 103 + -O6 104 + -DSQLITE_ENABLE_FTS4=1 105 + -DSQLITE_ENABLE_RTREE=1 106 + -DSQLITE_ENABLE_STAT4 107 + -DSQLITE_ENABLE_RBU 108 + -DSQLITE_MAX_ATTACHED=125 109 + -DLONGDOUBLE_TYPE=double 110 + --enable-session 111 + } 112 + "Device-One" { 113 + -O2 114 + -DSQLITE_DEBUG=1 115 + -DSQLITE_DEFAULT_AUTOVACUUM=1 116 + -DSQLITE_DEFAULT_CACHE_SIZE=64 117 + -DSQLITE_DEFAULT_PAGE_SIZE=1024 118 + -DSQLITE_DEFAULT_TEMP_CACHE_SIZE=32 119 + -DSQLITE_DISABLE_LFS=1 120 + -DSQLITE_ENABLE_ATOMIC_WRITE=1 121 + -DSQLITE_ENABLE_IOTRACE=1 122 + -DSQLITE_ENABLE_MEMORY_MANAGEMENT=1 123 + -DSQLITE_MAX_PAGE_SIZE=4096 124 + -DSQLITE_OMIT_LOAD_EXTENSION=1 125 + -DSQLITE_OMIT_PROGRESS_CALLBACK=1 126 + -DSQLITE_OMIT_VIRTUALTABLE=1 127 + -DSQLITE_ENABLE_HIDDEN_COLUMNS 128 + -DSQLITE_TEMP_STORE=3 129 + --enable-json1 130 + } 131 + "Device-Two" { 132 + -DSQLITE_4_BYTE_ALIGNED_MALLOC=1 133 + -DSQLITE_DEFAULT_AUTOVACUUM=1 134 + -DSQLITE_DEFAULT_CACHE_SIZE=1000 135 + -DSQLITE_DEFAULT_LOCKING_MODE=0 136 + -DSQLITE_DEFAULT_PAGE_SIZE=1024 137 + -DSQLITE_DEFAULT_TEMP_CACHE_SIZE=1000 138 + -DSQLITE_DISABLE_LFS=1 139 + -DSQLITE_ENABLE_FTS3=1 140 + -DSQLITE_ENABLE_MEMORY_MANAGEMENT=1 141 + -DSQLITE_ENABLE_RTREE=1 142 + -DSQLITE_MAX_COMPOUND_SELECT=50 143 + -DSQLITE_MAX_PAGE_SIZE=32768 144 + -DSQLITE_OMIT_TRACE=1 145 + -DSQLITE_TEMP_STORE=3 146 + -DSQLITE_THREADSAFE=2 147 + -DSQLITE_ENABLE_DESERIALIZE=1 148 + --enable-json1 --enable-fts5 --enable-session 149 + } 150 + "Locking-Style" { 151 + -O2 152 + -DSQLITE_ENABLE_LOCKING_STYLE=1 153 + } 154 + "Apple" { 155 + -Os 156 + -DHAVE_GMTIME_R=1 157 + -DHAVE_ISNAN=1 158 + -DHAVE_LOCALTIME_R=1 159 + -DHAVE_PREAD=1 160 + -DHAVE_PWRITE=1 161 + -DHAVE_USLEEP=1 162 + -DHAVE_USLEEP=1 163 + -DHAVE_UTIME=1 164 + -DSQLITE_DEFAULT_CACHE_SIZE=1000 165 + -DSQLITE_DEFAULT_CKPTFULLFSYNC=1 166 + -DSQLITE_DEFAULT_MEMSTATUS=1 167 + -DSQLITE_DEFAULT_PAGE_SIZE=1024 168 + -DSQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS=1 169 + -DSQLITE_ENABLE_API_ARMOR=1 170 + -DSQLITE_ENABLE_AUTO_PROFILE=1 171 + -DSQLITE_ENABLE_FLOCKTIMEOUT=1 172 + -DSQLITE_ENABLE_FTS3=1 173 + -DSQLITE_ENABLE_FTS3_PARENTHESIS=1 174 + -DSQLITE_ENABLE_FTS3_TOKENIZER=1 175 + if:os=="Darwin" -DSQLITE_ENABLE_LOCKING_STYLE=1 176 + -DSQLITE_ENABLE_PERSIST_WAL=1 177 + -DSQLITE_ENABLE_PURGEABLE_PCACHE=1 178 + -DSQLITE_ENABLE_RTREE=1 179 + -DSQLITE_ENABLE_SNAPSHOT=1 180 + # -DSQLITE_ENABLE_SQLLOG=1 181 + -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1 182 + -DSQLITE_MAX_LENGTH=2147483645 183 + -DSQLITE_MAX_VARIABLE_NUMBER=500000 184 + # -DSQLITE_MEMDEBUG=1 185 + -DSQLITE_NO_SYNC=1 186 + -DSQLITE_OMIT_AUTORESET=1 187 + -DSQLITE_OMIT_LOAD_EXTENSION=1 188 + -DSQLITE_PREFER_PROXY_LOCKING=1 189 + -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 190 + -DSQLITE_THREADSAFE=2 191 + -DSQLITE_USE_URI=1 192 + -DSQLITE_WRITE_WALFRAME_PREBUFFERED=1 193 + -DUSE_GUARDED_FD=1 194 + -DUSE_PREAD=1 195 + --enable-json1 --enable-fts5 196 + } 197 + "Extra-Robustness" { 198 + -DSQLITE_ENABLE_OVERSIZE_CELL_CHECK=1 199 + -DSQLITE_MAX_ATTACHED=62 200 + } 201 + "Devkit" { 202 + -DSQLITE_DEFAULT_FILE_FORMAT=4 203 + -DSQLITE_MAX_ATTACHED=30 204 + -DSQLITE_ENABLE_COLUMN_METADATA 205 + -DSQLITE_ENABLE_FTS4 206 + -DSQLITE_ENABLE_FTS5 207 + -DSQLITE_ENABLE_FTS4_PARENTHESIS 208 + -DSQLITE_DISABLE_FTS4_DEFERRED 209 + -DSQLITE_ENABLE_RTREE 210 + --enable-json1 --enable-fts5 211 + } 212 + "No-lookaside" { 213 + -DSQLITE_TEST_REALLOC_STRESS=1 214 + -DSQLITE_OMIT_LOOKASIDE=1 215 + -DHAVE_USLEEP=1 216 + } 217 + "Valgrind" { 218 + -DSQLITE_ENABLE_STAT4 219 + -DSQLITE_ENABLE_FTS4 220 + -DSQLITE_ENABLE_RTREE 221 + -DSQLITE_ENABLE_HIDDEN_COLUMNS 222 + --enable-json1 223 + } 224 + 225 + # The next group of configurations are used only by the 226 + # Failure-Detection platform. They are all the same, but we need 227 + # different names for them all so that they results appear in separate 228 + # subdirectories. 229 + # 230 + Fail0 {-O0} 231 + Fail2 {-O0} 232 + Fail3 {-O0} 233 + Fail4 {-O0} 234 + FuzzFail1 {-O0} 235 + FuzzFail2 {-O0} 236 +}] 237 + 238 +array set ::Platforms [strip_comments { 239 + Linux-x86_64 { 240 + "Check-Symbols" checksymbols 241 + "Fast-One" "fuzztest test" 242 + "Debug-One" "mptest test" 243 + "Have-Not" test 244 + "Secure-Delete" test 245 + "Unlock-Notify" "QUICKTEST_INCLUDE=notify2.test test" 246 + "User-Auth" tcltest 247 + "Update-Delete-Limit" test 248 + "Extra-Robustness" test 249 + "Device-Two" test 250 + "No-lookaside" test 251 + "Devkit" test 252 + "Apple" test 253 + "Sanitize" {QUICKTEST_OMIT=func4.test,nan.test test} 254 + "Device-One" fulltest 255 + "Default" "threadtest fulltest" 256 + "Valgrind" valgrindtest 257 + } 258 + Linux-i686 { 259 + "Devkit" test 260 + "Have-Not" test 261 + "Unlock-Notify" "QUICKTEST_INCLUDE=notify2.test test" 262 + "Device-One" test 263 + "Device-Two" test 264 + "Default" "threadtest fulltest" 265 + } 266 + Darwin-i386 { 267 + "Locking-Style" "mptest test" 268 + "Have-Not" test 269 + "Apple" "threadtest fulltest" 270 + } 271 + Darwin-x86_64 { 272 + "Locking-Style" "mptest test" 273 + "Have-Not" test 274 + "Apple" "threadtest fulltest" 275 + } 276 + "Windows NT-intel" { 277 + "Stdcall" test 278 + "Have-Not" test 279 + "Default" "mptest fulltestonly" 280 + } 281 + "Windows NT-amd64" { 282 + "Stdcall" test 283 + "Have-Not" test 284 + "Default" "mptest fulltestonly" 285 + } 286 + 287 + # The Failure-Detection platform runs various tests that deliberately 288 + # fail. This is used as a test of this script to verify that this script 289 + # correctly identifies failures. 290 + # 291 + Failure-Detection { 292 + Fail0 "TEST_FAILURE=0 test" 293 + Sanitize "TEST_FAILURE=1 test" 294 + Fail2 "TEST_FAILURE=2 valgrindtest" 295 + Fail3 "TEST_FAILURE=3 valgrindtest" 296 + Fail4 "TEST_FAILURE=4 test" 297 + FuzzFail1 "TEST_FAILURE=5 test" 298 + FuzzFail2 "TEST_FAILURE=5 valgrindtest" 299 + } 300 +}] 301 + 302 +proc make_test_suite {msvc withtcl name testtarget config} { 303 + 304 + # Tcl variable $opts is used to build up the value used to set the 305 + # OPTS Makefile variable. Variable $cflags holds the value for 306 + # CFLAGS. The makefile will pass OPTS to both gcc and lemon, but 307 + # CFLAGS is only passed to gcc. 308 + # 309 + set makeOpts "" 310 + set cflags [expr {$msvc ? "-Zi" : "-g"}] 311 + set opts "" 312 + set title ${name}($testtarget) 313 + set configOpts $withtcl 314 + set skip 0 315 + 316 + regsub -all {#[^\n]*\n} $config \n config 317 + foreach arg $config { 318 + if {$skip} { 319 + set skip 0 320 + continue 321 + } 322 + if {[regexp {^-[UD]} $arg]} { 323 + lappend opts $arg 324 + } elseif {[regexp {^[A-Z]+=} $arg]} { 325 + lappend testtarget $arg 326 + } elseif {[regexp {^if:([a-z]+)(.*)} $arg all key tail]} { 327 + # Arguments of the form 'if:os=="Linux"' will cause the subsequent 328 + # argument to be skipped if the $tcl_platform(os) is not "Linux", for 329 + # example... 330 + set skip [expr !(\$::tcl_platform($key)$tail)] 331 + } elseif {[regexp {^--(enable|disable)-} $arg]} { 332 + if {$msvc} { 333 + if {$arg eq "--disable-amalgamation"} { 334 + lappend makeOpts USE_AMALGAMATION=0 335 + continue 336 + } 337 + if {$arg eq "--disable-shared"} { 338 + lappend makeOpts USE_CRT_DLL=0 DYNAMIC_SHELL=0 339 + continue 340 + } 341 + if {$arg eq "--enable-fts5"} { 342 + lappend opts -DSQLITE_ENABLE_FTS5 343 + continue 344 + } 345 + if {$arg eq "--enable-json1"} { 346 + lappend opts -DSQLITE_ENABLE_JSON1 347 + continue 348 + } 349 + if {$arg eq "--enable-shared"} { 350 + lappend makeOpts USE_CRT_DLL=1 DYNAMIC_SHELL=1 351 + continue 352 + } 353 + } 354 + lappend configOpts $arg 355 + } else { 356 + if {$msvc} { 357 + if {$arg eq "-g"} { 358 + lappend cflags -Zi 359 + continue 360 + } 361 + if {[regexp -- {^-O(\d+)$} $arg all level]} then { 362 + lappend makeOpts OPTIMIZATIONS=$level 363 + continue 364 + } 365 + } 366 + lappend cflags $arg 367 + } 368 + } 369 + 370 + # Disable sync to make testing faster. 371 + # 372 + lappend opts -DSQLITE_NO_SYNC=1 373 + 374 + # Some configurations already set HAVE_USLEEP; in that case, skip it. 375 + # 376 + if {[lsearch -regexp $opts {^-DHAVE_USLEEP(?:=|$)}]==-1} { 377 + lappend opts -DHAVE_USLEEP=1 378 + } 379 + 380 + # Add the define for this platform. 381 + # 382 + if {$::tcl_platform(platform)=="windows"} { 383 + lappend opts -DSQLITE_OS_WIN=1 384 + } else { 385 + lappend opts -DSQLITE_OS_UNIX=1 386 + } 387 + 388 + # Set the sub-directory to use. 389 + # 390 + set dir [string tolower [string map {- _ " " _ "(" _ ")" _} $name]] 391 + 392 + # Join option lists into strings, using space as delimiter. 393 + # 394 + set makeOpts [join $makeOpts " "] 395 + set cflags [join $cflags " "] 396 + set opts [join $opts " "] 397 + 398 + return [list $title $dir $configOpts $testtarget $makeOpts $cflags $opts] 399 +} 400 + 401 +# Configuration verification: Check that each entry in the list of configs 402 +# specified for each platforms exists. 403 +# 404 +foreach {key value} [array get ::Platforms] { 405 + foreach {v t} $value { 406 + if {0==[info exists ::Configs($v)]} { 407 + puts stderr "No such configuration: \"$v\"" 408 + exit -1 409 + } 410 + } 411 +} 412 +
Changes to test/shell1.test.
1018 1018 # command channels opened for it as textual ones), the carriage 1019 1019 # return character (and on Windows, the end-of-file character) 1020 1020 # cannot be used here. 1021 1021 # 1022 1022 if {$i==0x0D || ($tcl_platform(platform)=="windows" && $i==0x1A)} { 1023 1023 continue 1024 1024 } 1025 + # Tcl 8.7 maps 0x80 through 0x9f into valid UTF8. So skip those tests. 1026 + if {$i>=0x80 && $i<=0x9f} continue 1025 1027 if {$i>=0xE0 && $tcl_platform(os)=="OpenBSD"} continue 1026 1028 if {$i>=0xE0 && $i<=0xEF && $tcl_platform(os)=="Linux"} continue 1027 1029 set hex [format %02X $i] 1028 1030 set char [subst \\x$hex]; set oldChar $char 1029 1031 set escapes [list] 1030 1032 if {$tcl_platform(platform)=="windows"} { 1031 1033 #
Changes to test/skipscan1.test.
340 340 341 341 optimization_control db skip-scan 0 342 342 do_execsql_test skipscan1-9.3 { 343 343 EXPLAIN QUERY PLAN 344 344 SELECT * FROM t9a WHERE b IN (SELECT x FROM t9b WHERE y!=5); 345 345 } {/{SCAN TABLE t9a}/} 346 346 optimization_control db skip-scan 1 347 + 348 +do_execsql_test skipscan1-2.1 { 349 + CREATE TABLE t6(a TEXT, b INT, c INT, d INT); 350 + CREATE INDEX t6abc ON t6(a,b,c); 351 + INSERT INTO t6 VALUES('abc',123,4,5); 352 + 353 + ANALYZE; 354 + DELETE FROM sqlite_stat1; 355 + INSERT INTO sqlite_stat1 VALUES('t6','t6abc','10000 5000 2000 10'); 356 + ANALYZE sqlite_master; 357 + DELETE FROM t6; 358 +} {} 359 + 360 +do_execsql_test skipscan1-2.2eqp { 361 + EXPLAIN QUERY PLAN 362 + SELECT a,b,c,d,'|' FROM t6 WHERE d<>99 AND b=345 ORDER BY a; 363 +} {/* USING INDEX t6abc (ANY(a) AND b=?)*/} 364 +do_execsql_test skipscan1-2.2 { 365 + SELECT a,b,c,d,'|' FROM t6 WHERE d<>99 AND b=345 ORDER BY a; 366 +} {} 367 + 368 +do_execsql_test skipscan1-2.3eqp { 369 + EXPLAIN QUERY PLAN 370 + SELECT a,b,c,d,'|' FROM t6 WHERE d<>99 AND b=345 ORDER BY a DESC; 371 +} {/* USING INDEX t6abc (ANY(a) AND b=?)*/} 372 +do_execsql_test skipscan1-2.3 { 373 + SELECT a,b,c,d,'|' FROM t6 WHERE d<>99 AND b=345 ORDER BY a DESC; 374 +} {} 347 375 348 376 finish_test
Changes to test/sqllimits1.test.
885 885 )))) 886 886 } "1 {too many columns in result set}" 887 887 888 888 889 889 foreach {key value} [array get saved] { 890 890 catch {set $key $value} 891 891 } 892 + 893 +#------------------------------------------------------------------------- 894 +# At one point the following caused an assert() to fail. 895 +# 896 +sqlite3_limit db SQLITE_LIMIT_LENGTH 10000 897 +set nm [string repeat x 10000] 898 +do_catchsql_test sqllimits1-17.1 " 899 + CREATE TABLE $nm (x PRIMARY KEY) 900 +" {1 {string or blob too big}} 901 + 892 902 finish_test
Changes to test/wal2.test.
1105 1105 2 00666 1106 1106 3 00600 1107 1107 4 00755 1108 1108 } { 1109 1109 set effective [format %.5o [expr $permissions & ~$umask]] 1110 1110 do_test wal2-12.2.$tn.1 { 1111 1111 file attributes test.db -permissions $permissions 1112 - file attributes test.db -permissions 1112 + string map {o 0} [file attributes test.db -permissions] 1113 1113 } $permissions 1114 1114 do_test wal2-12.2.$tn.2 { 1115 1115 list [file exists test.db-wal] [file exists $shmpath] 1116 1116 } {0 0} 1117 1117 do_test wal2-12.2.$tn.3 { 1118 1118 sqlite3 db test.db 1119 1119 execsql { INSERT INTO tx DEFAULT VALUES } 1120 1120 list [file exists test.db-wal] [file exists $shmpath] 1121 1121 } {1 1} 1122 1122 do_test wal2-12.2.$tn.4 { 1123 - list [file attr test.db-wal -perm] [file attr $shmpath -perm] 1123 + set x [list [file attr test.db-wal -perm] [file attr $shmpath -perm]] 1124 + string map {o 0} $x 1124 1125 } [list $effective $effective] 1125 1126 do_test wal2-12.2.$tn.5 { 1126 1127 ifcapable enable_persist_wal { 1127 1128 file_control_persist_wal db 0 1128 1129 } 1129 1130 db close 1130 1131 list [file exists test.db-wal] [file exists $shmpath] ................................................................................ 1187 1188 file attr test.db -perm $db_perm 1188 1189 file attr test.db-wal -perm $wal_perm 1189 1190 file attr $shmpath -perm $shm_perm 1190 1191 1191 1192 set L [file attr test.db -perm] 1192 1193 lappend L [file attr test.db-wal -perm] 1193 1194 lappend L [file attr $shmpath -perm] 1195 + string map {o 0} $L 1194 1196 } [list $db_perm $wal_perm $shm_perm] 1195 1197 1196 1198 # If $can_open is true, then it should be possible to open a database 1197 1199 # handle. Otherwise, if $can_open is 0, attempting to open the db 1198 1200 # handle throws an "unable to open database file" exception. 1199 1201 # 1200 1202 set r(1) {0 ok}
Added test/wapp.tcl.
1 +# Copyright (c) 2017 D. Richard Hipp 2 +# 3 +# This program is free software; you can redistribute it and/or 4 +# modify it under the terms of the Simplified BSD License (also 5 +# known as the "2-Clause License" or "FreeBSD License".) 6 +# 7 +# This program is distributed in the hope that it will be useful, 8 +# but without any warranty; without even the implied warranty of 9 +# merchantability or fitness for a particular purpose. 10 +# 11 +#--------------------------------------------------------------------------- 12 +# 13 +# Design rules: 14 +# 15 +# (1) All identifiers in the global namespace begin with "wapp" 16 +# 17 +# (2) Indentifiers intended for internal use only begin with "wappInt" 18 +# 19 +package require Tcl 8.6 20 + 21 +# Add text to the end of the HTTP reply. No interpretation or transformation 22 +# of the text is performs. The argument should be enclosed within {...} 23 +# 24 +proc wapp {txt} { 25 + global wapp 26 + dict append wapp .reply $txt 27 +} 28 + 29 +# Add text to the page under construction. Do no escaping on the text. 30 +# 31 +# Though "unsafe" in general, there are uses for this kind of thing. 32 +# For example, if you want to return the complete, unmodified content of 33 +# a file: 34 +# 35 +# set fd [open content.html rb] 36 +# wapp-unsafe [read $fd] 37 +# close $fd 38 +# 39 +# You could do the same thing using ordinary "wapp" instead of "wapp-unsafe". 40 +# The difference is that wapp-safety-check will complain about the misuse 41 +# of "wapp", but it assumes that the person who write "wapp-unsafe" understands 42 +# the risks. 43 +# 44 +# Though occasionally necessary, the use of this interface should be minimized. 45 +# 46 +proc wapp-unsafe {txt} { 47 + global wapp 48 + dict append wapp .reply $txt 49 +} 50 + 51 +# Add text to the end of the reply under construction. The following 52 +# substitutions are made: 53 +# 54 +# %html(...) Escape text for inclusion in HTML 55 +# %url(...) Escape text for use as a URL 56 +# %qp(...) Escape text for use as a URI query parameter 57 +# %string(...) Escape text for use within a JSON string 58 +# %unsafe(...) No transformations of the text 59 +# 60 +# The substitutions above terminate at the first ")" character. If the 61 +# text of the TCL string in ... contains ")" characters itself, use instead: 62 +# 63 +# %html%(...)% 64 +# %url%(...)% 65 +# %qp%(...)% 66 +# %string%(...)% 67 +# %unsafe%(...)% 68 +# 69 +# In other words, use "%(...)%" instead of "(...)" to include the TCL string 70 +# to substitute. 71 +# 72 +# The %unsafe substitution should be avoided whenever possible, obviously. 73 +# In addition to the substitutions above, the text also does backslash 74 +# escapes. 75 +# 76 +# The wapp-trim proc works the same as wapp-subst except that it also removes 77 +# whitespace from the left margin, so that the generated HTML/CSS/Javascript 78 +# does not appear to be indented when delivered to the client web browser. 79 +# 80 +if {$tcl_version>=8.7} { 81 + proc wapp-subst {txt} { 82 + global wapp 83 + regsub -all -command \ 84 + {%(html|url|qp|string|unsafe){1,1}?(|%)\((.+)\)\2} $txt wappInt-enc txt 85 + dict append wapp .reply [subst -novariables -nocommand $txt] 86 + } 87 + proc wapp-trim {txt} { 88 + global wapp 89 + regsub -all {\n\s+} [string trim $txt] \n txt 90 + regsub -all -command \ 91 + {%(html|url|qp|string|unsafe){1,1}?(|%)\((.+)\)\2} $txt wappInt-enc txt 92 + dict append wapp .reply [subst -novariables -nocommand $txt] 93 + } 94 + proc wappInt-enc {all mode nu1 txt} { 95 + return [uplevel 2 "wappInt-enc-$mode \"$txt\""] 96 + } 97 +} else { 98 + proc wapp-subst {txt} { 99 + global wapp 100 + regsub -all {%(html|url|qp|string|unsafe){1,1}?(|%)\((.+)\)\2} $txt \ 101 + {[wappInt-enc-\1 "\3"]} txt 102 + dict append wapp .reply [uplevel 1 [list subst -novariables $txt]] 103 + } 104 + proc wapp-trim {txt} { 105 + global wapp 106 + regsub -all {\n\s+} [string trim $txt] \n txt 107 + regsub -all {%(html|url|qp|string|unsafe){1,1}?(|%)\((.+)\)\2} $txt \ 108 + {[wappInt-enc-\1 "\3"]} txt 109 + dict append wapp .reply [uplevel 1 [list subst -novariables $txt]] 110 + } 111 +} 112 + 113 +# There must be a wappInt-enc-NAME routine for each possible substitution 114 +# in wapp-subst. Thus there are routines for "html", "url", "qp", and "unsafe". 115 +# 116 +# wappInt-enc-html Escape text so that it is safe to use in the 117 +# body of an HTML document. 118 +# 119 +# wappInt-enc-url Escape text so that it is safe to pass as an 120 +# argument to href= and src= attributes in HTML. 121 +# 122 +# wappInt-enc-qp Escape text so that it is safe to use as the 123 +# value of a query parameter in a URL or in 124 +# post data or in a cookie. 125 +# 126 +# wappInt-enc-string Escape ", ', \, and < for using inside of a 127 +# javascript string literal. The < character 128 +# is escaped to prevent "</script>" from causing 129 +# problems in embedded javascript. 130 +# 131 +# wappInt-enc-unsafe Perform no encoding at all. Unsafe. 132 +# 133 +proc wappInt-enc-html {txt} { 134 + return [string map {& & < < > > \" " \\ \} $txt] 135 +} 136 +proc wappInt-enc-unsafe {txt} { 137 + return $txt 138 +} 139 +proc wappInt-enc-url {s} { 140 + if {[regsub -all {[^-{}@~?=#_.:/a-zA-Z0-9]} $s {[wappInt-%HHchar {&}]} s]} { 141 + set s [subst -novar -noback $s] 142 + } 143 + if {[regsub -all {[{}]} $s {[wappInt-%HHchar \\&]} s]} { 144 + set s [subst -novar -noback $s] 145 + } 146 + return $s 147 +} 148 +proc wappInt-enc-qp {s} { 149 + if {[regsub -all {[^-{}_.a-zA-Z0-9]} $s {[wappInt-%HHchar {&}]} s]} { 150 + set s [subst -novar -noback $s] 151 + } 152 + if {[regsub -all {[{}]} $s {[wappInt-%HHchar \\&]} s]} { 153 + set s [subst -novar -noback $s] 154 + } 155 + return $s 156 +} 157 +proc wappInt-enc-string {s} { 158 + return [string map {\\ \\\\ \" \\\" ' \\' < \\u003c} $s] 159 +} 160 + 161 +# This is a helper routine for wappInt-enc-url and wappInt-enc-qp. It returns 162 +# an appropriate %HH encoding for the single character c. If c is a unicode 163 +# character, then this routine might return multiple bytes: %HH%HH%HH 164 +# 165 +proc wappInt-%HHchar {c} { 166 + if {$c==" "} {return +} 167 + return [regsub -all .. [binary encode hex [encoding convertto utf-8 $c]] {%&}] 168 +} 169 + 170 + 171 +# Undo the www-url-encoded format. 172 +# 173 +# HT: This code stolen from ncgi.tcl 174 +# 175 +proc wappInt-decode-url {str} { 176 + set str [string map [list + { } "\\" "\\\\" \[ \\\[ \] \\\]] $str] 177 + regsub -all -- \ 178 + {%([Ee][A-Fa-f0-9])%([89ABab][A-Fa-f0-9])%([89ABab][A-Fa-f0-9])} \ 179 + $str {[encoding convertfrom utf-8 [binary decode hex \1\2\3]]} str 180 + regsub -all -- \ 181 + {%([CDcd][A-Fa-f0-9])%([89ABab][A-Fa-f0-9])} \ 182 + $str {[encoding convertfrom utf-8 [binary decode hex \1\2]]} str 183 + regsub -all -- {%([0-7][A-Fa-f0-9])} $str {\\u00\1} str 184 + return [subst -novar $str] 185 +} 186 + 187 +# Reset the document back to an empty string. 188 +# 189 +proc wapp-reset {} { 190 + global wapp 191 + dict set wapp .reply {} 192 +} 193 + 194 +# Change the mime-type of the result document. 195 +# 196 +proc wapp-mimetype {x} { 197 + global wapp 198 + dict set wapp .mimetype $x 199 +} 200 + 201 +# Change the reply code. 202 +# 203 +proc wapp-reply-code {x} { 204 + global wapp 205 + dict set wapp .reply-code $x 206 +} 207 + 208 +# Set a cookie 209 +# 210 +proc wapp-set-cookie {name value} { 211 + global wapp 212 + dict lappend wapp .new-cookies $name $value 213 +} 214 + 215 +# Unset a cookie 216 +# 217 +proc wapp-clear-cookie {name} { 218 + wapp-set-cookie $name {} 219 +} 220 + 221 +# Add extra entries to the reply header 222 +# 223 +proc wapp-reply-extra {name value} { 224 + global wapp 225 + dict lappend wapp .reply-extra $name $value 226 +} 227 + 228 +# Specifies how the web-page under construction should be cached. 229 +# The argument should be one of: 230 +# 231 +# no-cache 232 +# max-age=N (for some integer number of seconds, N) 233 +# private,max-age=N 234 +# 235 +proc wapp-cache-control {x} { 236 + wapp-reply-extra Cache-Control $x 237 +} 238 + 239 +# Redirect to a different web page 240 +# 241 +proc wapp-redirect {uri} { 242 + wapp-reply-code {307 Redirect} 243 + wapp-reply-extra Location $uri 244 +} 245 + 246 +# Return the value of a wapp parameter 247 +# 248 +proc wapp-param {name {dflt {}}} { 249 + global wapp 250 + if {![dict exists $wapp $name]} {return $dflt} 251 + return [dict get $wapp $name] 252 +} 253 + 254 +# Return true if a and only if the wapp parameter $name exists 255 +# 256 +proc wapp-param-exists {name} { 257 + global wapp 258 + return [dict exists $wapp $name] 259 +} 260 + 261 +# Set the value of a wapp parameter 262 +# 263 +proc wapp-set-param {name value} { 264 + global wapp 265 + dict set wapp $name $value 266 +} 267 + 268 +# Return all parameter names that match the GLOB pattern, or all 269 +# names if the GLOB pattern is omitted. 270 +# 271 +proc wapp-param-list {{glob {*}}} { 272 + global wapp 273 + return [dict keys $wapp $glob] 274 +} 275 + 276 +# By default, Wapp does not decode query parameters and POST parameters 277 +# for cross-origin requests. This is a security restriction, designed to 278 +# help prevent cross-site request forgery (CSRF) attacks. 279 +# 280 +# As a consequence of this restriction, URLs for sites generated by Wapp 281 +# that contain query parameters will not work as URLs found in other 282 +# websites. You cannot create a link from a second website into a Wapp 283 +# website if the link contains query planner, by default. 284 +# 285 +# Of course, it is sometimes desirable to allow query parameters on external 286 +# links. For URLs for which this is safe, the application should invoke 287 +# wapp-allow-xorigin-params. This procedure tells Wapp that it is safe to 288 +# go ahead and decode the query parameters even for cross-site requests. 289 +# 290 +# In other words, for Wapp security is the default setting. Individual pages 291 +# need to actively disable the cross-site request security if those pages 292 +# are safe for cross-site access. 293 +# 294 +proc wapp-allow-xorigin-params {} { 295 + global wapp 296 + if {![dict exists $wapp .qp] && ![dict get $wapp SAME_ORIGIN]} { 297 + wappInt-decode-query-params 298 + } 299 +} 300 + 301 +# Set the content-security-policy. 302 +# 303 +# The default content-security-policy is very strict: "default-src 'self'" 304 +# The default policy prohibits the use of in-line javascript or CSS. 305 +# 306 +# Provide an alternative CSP as the argument. Or use "off" to disable 307 +# the CSP completely. 308 +# 309 +proc wapp-content-security-policy {val} { 310 + global wapp 311 + if {$val=="off"} { 312 + dict unset wapp .csp 313 + } else { 314 + dict set wapp .csp $val 315 + } 316 +} 317 + 318 +# Examine the bodys of all procedures in this program looking for 319 +# unsafe calls to various Wapp interfaces. Return a text string 320 +# containing warnings. Return an empty string if all is ok. 321 +# 322 +# This routine is advisory only. It misses some constructs that are 323 +# dangerous and flags others that are safe. 324 +# 325 +proc wapp-safety-check {} { 326 + set res {} 327 + foreach p [info procs] { 328 + set ln 0 329 + foreach x [split [info body $p] \n] { 330 + incr ln 331 + if {[regexp {^[ \t]*wapp[ \t]+([^\n]+)} $x all tail] 332 + && [string index $tail 0]!="\173" 333 + && [regexp {[[$]} $tail] 334 + } { 335 + append res "$p:$ln: unsafe \"wapp\" call: \"[string trim $x]\"\n" 336 + } 337 + if {[regexp {^[ \t]*wapp-(subst|trim)[ \t]+[^\173]} $x all cx]} { 338 + append res "$p:$ln: unsafe \"wapp-$cx\" call: \"[string trim $x]\"\n" 339 + } 340 + } 341 + } 342 + return $res 343 +} 344 + 345 +# Return a string that descripts the current environment. Applications 346 +# might find this useful for debugging. 347 +# 348 +proc wapp-debug-env {} { 349 + global wapp 350 + set out {} 351 + foreach var [lsort [dict keys $wapp]] { 352 + if {[string index $var 0]=="."} continue 353 + append out "$var = [list [dict get $wapp $var]]\n" 354 + } 355 + append out "\[pwd\] = [list [pwd]]\n" 356 + return $out 357 +} 358 + 359 +# Tracing function for each HTTP request. This is overridden by wapp-start 360 +# if tracing is enabled. 361 +# 362 +proc wappInt-trace {} {} 363 + 364 +# Start up a listening socket. Arrange to invoke wappInt-new-connection 365 +# for each inbound HTTP connection. 366 +# 367 +# port Listen on this TCP port. 0 means to select a port 368 +# that is not currently in use 369 +# 370 +# wappmode One of "scgi", "remote-scgi", "server", or "local". 371 +# 372 +# fromip If not {}, then reject all requests from IP addresses 373 +# other than $fromip 374 +# 375 +proc wappInt-start-listener {port wappmode fromip} { 376 + if {[string match *scgi $wappmode]} { 377 + set type SCGI 378 + set server [list wappInt-new-connection \ 379 + wappInt-scgi-readable $wappmode $fromip] 380 + } else { 381 + set type HTTP 382 + set server [list wappInt-new-connection \ 383 + wappInt-http-readable $wappmode $fromip] 384 + } 385 + if {$wappmode=="local" || $wappmode=="scgi"} { 386 + set x [socket -server $server -myaddr 127.0.0.1 $port] 387 + } else { 388 + set x [socket -server $server $port] 389 + } 390 + set coninfo [chan configure $x -sockname] 391 + set port [lindex $coninfo 2] 392 + if {$wappmode=="local"} { 393 + wappInt-start-browser http://127.0.0.1:$port/ 394 + } elseif {$fromip!=""} { 395 + puts "Listening for $type requests on TCP port $port from IP $fromip" 396 + } else { 397 + puts "Listening for $type requests on TCP port $port" 398 + } 399 +} 400 + 401 +# Start a web-browser and point it at $URL 402 +# 403 +proc wappInt-start-browser {url} { 404 + global tcl_platform 405 + if {$tcl_platform(platform)=="windows"} { 406 + exec cmd /c start $url & 407 + } elseif {$tcl_platform(os)=="Darwin"} { 408 + exec open $url & 409 + } elseif {[catch {exec xdg-open $url}]} { 410 + exec firefox $url & 411 + } 412 +} 413 + 414 +# This routine is a "socket -server" callback. The $chan, $ip, and $port 415 +# arguments are added by the socket command. 416 +# 417 +# Arrange to invoke $callback when content is available on the new socket. 418 +# The $callback will process inbound HTTP or SCGI content. Reject the 419 +# request if $fromip is not an empty string and does not match $ip. 420 +# 421 +proc wappInt-new-connection {callback wappmode fromip chan ip port} { 422 + upvar #0 wappInt-$chan W 423 + if {$fromip!="" && ![string match $fromip $ip]} { 424 + close $chan 425 + return 426 + } 427 + set W [dict create REMOTE_ADDR $ip REMOTE_PORT $port WAPP_MODE $wappmode \ 428 + .header {}] 429 + fconfigure $chan -blocking 0 -translation binary 430 + fileevent $chan readable [list $callback $chan] 431 +} 432 + 433 +# Close an input channel 434 +# 435 +proc wappInt-close-channel {chan} { 436 + if {$chan=="stdout"} { 437 + # This happens after completing a CGI request 438 + exit 0 439 + } else { 440 + unset ::wappInt-$chan 441 + close $chan 442 + } 443 +} 444 + 445 +# Process new text received on an inbound HTTP request 446 +# 447 +proc wappInt-http-readable {chan} { 448 + if {[catch [list wappInt-http-readable-unsafe $chan] msg]} { 449 + puts stderr "$msg\n$::errorInfo" 450 + wappInt-close-channel $chan 451 + } 452 +} 453 +proc wappInt-http-readable-unsafe {chan} { 454 + upvar #0 wappInt-$chan W wapp wapp 455 + if {![dict exists $W .toread]} { 456 + # If the .toread key is not set, that means we are still reading 457 + # the header 458 + set line [string trimright [gets $chan]] 459 + set n [string length $line] 460 + if {$n>0} { 461 + if {[dict get $W .header]=="" || [regexp {^\s+} $line]} { 462 + dict append W .header $line 463 + } else { 464 + dict append W .header \n$line 465 + } 466 + if {[string length [dict get $W .header]]>100000} { 467 + error "HTTP request header too big - possible DOS attack" 468 + } 469 + } elseif {$n==0} { 470 + # We have reached the blank line that terminates the header. 471 + global argv0 472 + set a0 [file normalize $argv0] 473 + dict set W SCRIPT_FILENAME $a0 474 + dict set W DOCUMENT_ROOT [file dir $a0] 475 + if {[wappInt-parse-header $chan]} { 476 + catch {close $chan} 477 + return 478 + } 479 + set len 0 480 + if {[dict exists $W CONTENT_LENGTH]} { 481 + set len [dict get $W CONTENT_LENGTH] 482 + } 483 + if {$len>0} { 484 + # Still need to read the query content 485 + dict set W .toread $len 486 + } else { 487 + # There is no query content, so handle the request immediately 488 + set wapp $W 489 + wappInt-handle-request $chan 0 490 + } 491 + } 492 + } else { 493 + # If .toread is set, that means we are reading the query content. 494 + # Continue reading until .toread reaches zero. 495 + set got [read $chan [dict get $W .toread]] 496 + dict append W CONTENT $got 497 + dict set W .toread [expr {[dict get $W .toread]-[string length $got]}] 498 + if {[dict get $W .toread]<=0} { 499 + # Handle the request as soon as all the query content is received 500 + set wapp $W 501 + wappInt-handle-request $chan 0 502 + } 503 + } 504 +} 505 + 506 +# Decode the HTTP request header. 507 +# 508 +# This routine is always running inside of a [catch], so if 509 +# any problems arise, simply raise an error. 510 +# 511 +proc wappInt-parse-header {chan} { 512 + upvar #0 wappInt-$chan W 513 + set hdr [split [dict get $W .header] \n] 514 + if {$hdr==""} {return 1} 515 + set req [lindex $hdr 0] 516 + dict set W REQUEST_METHOD [set method [lindex $req 0]] 517 + if {[lsearch {GET HEAD POST} $method]<0} { 518 + error "unsupported request method: \"[dict get $W REQUEST_METHOD]\"" 519 + } 520 + set uri [lindex $req 1] 521 + set split_uri [split $uri ?] 522 + set uri0 [lindex $split_uri 0] 523 + if {![regexp {^/[-.a-z0-9_/]*$} $uri0]} { 524 + error "invalid request uri: \"$uri0\"" 525 + } 526 + dict set W REQUEST_URI $uri0 527 + dict set W PATH_INFO $uri0 528 + set uri1 [lindex $split_uri 1] 529 + dict set W QUERY_STRING $uri1 530 + set n [llength $hdr] 531 + for {set i 1} {$i<$n} {incr i} { 532 + set x [lindex $hdr $i] 533 + if {![regexp {^(.+): +(.*)$} $x all name value]} { 534 + error "invalid header line: \"$x\"" 535 + } 536 + set name [string toupper $name] 537 + switch -- $name { 538 + REFERER {set name HTTP_REFERER} 539 + USER-AGENT {set name HTTP_USER_AGENT} 540 + CONTENT-LENGTH {set name CONTENT_LENGTH} 541 + CONTENT-TYPE {set name CONTENT_TYPE} 542 + HOST {set name HTTP_HOST} 543 + COOKIE {set name HTTP_COOKIE} 544 + ACCEPT-ENCODING {set name HTTP_ACCEPT_ENCODING} 545 + default {set name .hdr:$name} 546 + } 547 + dict set W $name $value 548 + } 549 + return 0 550 +} 551 + 552 +# Decode the QUERY_STRING parameters from a GET request or the 553 +# application/x-www-form-urlencoded CONTENT from a POST request. 554 +# 555 +# This routine sets the ".qp" element of the ::wapp dict as a signal 556 +# that query parameters have already been decoded. 557 +# 558 +proc wappInt-decode-query-params {} { 559 + global wapp 560 + dict set wapp .qp 1 561 + if {[dict exists $wapp QUERY_STRING]} { 562 + foreach qterm [split [dict get $wapp QUERY_STRING] &] { 563 + set qsplit [split $qterm =] 564 + set nm [lindex $qsplit 0] 565 + if {[regexp {^[a-z][a-z0-9]*$} $nm]} { 566 + dict set wapp $nm [wappInt-decode-url [lindex $qsplit 1]] 567 + } 568 + } 569 + } 570 + if {[dict exists $wapp CONTENT_TYPE] && [dict exists $wapp CONTENT]} { 571 + set ctype [dict get $wapp CONTENT_TYPE] 572 + if {$ctype=="application/x-www-form-urlencoded"} { 573 + foreach qterm [split [string trim [dict get $wapp CONTENT]] &] { 574 + set qsplit [split $qterm =] 575 + set nm [lindex $qsplit 0] 576 + if {[regexp {^[a-z][-a-z0-9_]*$} $nm]} { 577 + dict set wapp $nm [wappInt-decode-url [lindex $qsplit 1]] 578 + } 579 + } 580 + } elseif {[string match multipart/form-data* $ctype]} { 581 + regexp {^(.*?)\r\n(.*)$} [dict get $wapp CONTENT] all divider body 582 + set ndiv [string length $divider] 583 + while {[string length $body]} { 584 + set idx [string first $divider $body] 585 + set unit [string range $body 0 [expr {$idx-3}]] 586 + set body [string range $body [expr {$idx+$ndiv+2}] end] 587 + if {[regexp {^Content-Disposition: form-data; (.*?)\r\n\r\n(.*)$} \ 588 + $unit unit hdr content]} { 589 + if {[regexp {name="(.*)"; filename="(.*)"\r\nContent-Type: (.*?)$}\ 590 + $hdr hr name filename mimetype]} { 591 + dict set wapp $name.filename \ 592 + [string map [list \\\" \" \\\\ \\] $filename] 593 + dict set wapp $name.mimetype $mimetype 594 + dict set wapp $name.content $content 595 + } elseif {[regexp {name="(.*)"} $hdr hr name]} { 596 + dict set wapp $name $content 597 + } 598 + } 599 + } 600 + } 601 + } 602 +} 603 + 604 +# Invoke application-supplied methods to generate a reply to 605 +# a single HTTP request. 606 +# 607 +# This routine always runs within [catch], so handle exceptions by 608 +# invoking [error]. 609 +# 610 +proc wappInt-handle-request {chan useCgi} { 611 + global wapp 612 + dict set wapp .reply {} 613 + dict set wapp .mimetype {text/html; charset=utf-8} 614 + dict set wapp .reply-code {200 Ok} 615 + dict set wapp .csp {default-src 'self'} 616 + 617 + # Set up additional CGI environment values 618 + # 619 + if {![dict exists $wapp HTTP_HOST]} { 620 + dict set wapp BASE_URL {} 621 + } elseif {[dict exists $wapp HTTPS]} { 622 + dict set wapp BASE_URL https://[dict get $wapp HTTP_HOST] 623 + } else { 624 + dict set wapp BASE_URL http://[dict get $wapp HTTP_HOST] 625 + } 626 + if {![dict exists $wapp REQUEST_URI]} { 627 + dict set wapp REQUEST_URI / 628 + } elseif {[regsub {\?.*} [dict get $wapp REQUEST_URI] {} newR]} { 629 + # Some servers (ex: nginx) append the query parameters to REQUEST_URI. 630 + # These need to be stripped off 631 + dict set wapp REQUEST_URI $newR 632 + } 633 + if {[dict exists $wapp SCRIPT_NAME]} { 634 + dict append wapp BASE_URL [dict get $wapp SCRIPT_NAME] 635 + } else { 636 + dict set wapp SCRIPT_NAME {} 637 + } 638 + if {![dict exists $wapp PATH_INFO]} { 639 + # If PATH_INFO is missing (ex: nginx) then construct it 640 + set URI [dict get $wapp REQUEST_URI] 641 + set skip [string length [dict get $wapp SCRIPT_NAME]] 642 + dict set wapp PATH_INFO [string range $URI $skip end] 643 + } 644 + if {[regexp {^/([^/]+)(.*)$} [dict get $wapp PATH_INFO] all head tail]} { 645 + dict set wapp PATH_HEAD $head 646 + dict set wapp PATH_TAIL [string trimleft $tail /] 647 + } else { 648 + dict set wapp PATH_INFO {} 649 + dict set wapp PATH_HEAD {} 650 + dict set wapp PATH_TAIL {} 651 + } 652 + dict set wapp SELF_URL [dict get $wapp BASE_URL]/[dict get $wapp PATH_HEAD] 653 + 654 + # Parse query parameters from the query string, the cookies, and 655 + # POST data 656 + # 657 + if {[dict exists $wapp HTTP_COOKIE]} { 658 + foreach qterm [split [dict get $wapp HTTP_COOKIE] {;}] { 659 + set qsplit [split [string trim $qterm] =] 660 + set nm [lindex $qsplit 0] 661 + if {[regexp {^[a-z][-a-z0-9_]*$} $nm]} { 662 + dict set wapp $nm [wappInt-decode-url [lindex $qsplit 1]] 663 + } 664 + } 665 + } 666 + set same_origin 0 667 + if {[dict exists $wapp HTTP_REFERER]} { 668 + set referer [dict get $wapp HTTP_REFERER] 669 + set base [dict get $wapp BASE_URL] 670 + if {$referer==$base || [string match $base/* $referer]} { 671 + set same_origin 1 672 + } 673 + } 674 + dict set wapp SAME_ORIGIN $same_origin 675 + if {$same_origin} { 676 + wappInt-decode-query-params 677 + } 678 + 679 + # Invoke the application-defined handler procedure for this page 680 + # request. If an error occurs while running that procedure, generate 681 + # an HTTP reply that contains the error message. 682 + # 683 + wapp-before-dispatch-hook 684 + wappInt-trace 685 + set mname [dict get $wapp PATH_HEAD] 686 + if {[catch { 687 + if {$mname!="" && [llength [info proc wapp-page-$mname]]>0} { 688 + wapp-page-$mname 689 + } else { 690 + wapp-default 691 + } 692 + } msg]} { 693 + if {[wapp-param WAPP_MODE]=="local" || [wapp-param WAPP_MODE]=="server"} { 694 + puts "ERROR: $::errorInfo" 695 + } 696 + wapp-reset 697 + wapp-reply-code "500 Internal Server Error" 698 + wapp-mimetype text/html 699 + wapp-trim { 700 + <h1>Wapp Application Error</h1> 701 + <pre>%html($::errorInfo)</pre> 702 + } 703 + dict unset wapp .new-cookies 704 + } 705 + 706 + # Transmit the HTTP reply 707 + # 708 + if {$chan=="stdout"} { 709 + puts $chan "Status: [dict get $wapp .reply-code]\r" 710 + } else { 711 + puts $chan "HTTP/1.1 [dict get $wapp .reply-code]\r" 712 + puts $chan "Server: wapp\r" 713 + puts $chan "Connection: close\r" 714 + } 715 + if {[dict exists $wapp .reply-extra]} { 716 + foreach {name value} [dict get $wapp .reply-extra] { 717 + puts $chan "$name: $value\r" 718 + } 719 + } 720 + if {[dict exists $wapp .csp]} { 721 + puts $chan "Content-Security-Policy: [dict get $wapp .csp]\r" 722 + } 723 + set mimetype [dict get $wapp .mimetype] 724 + puts $chan "Content-Type: $mimetype\r" 725 + if {[dict exists $wapp .new-cookies]} { 726 + foreach {nm val} [dict get $wapp .new-cookies] { 727 + if {[regexp {^[a-z][-a-z0-9_]*$} $nm]} { 728 + if {$val==""} { 729 + puts $chan "Set-Cookie: $nm=; HttpOnly; Path=/; Max-Age=1\r" 730 + } else { 731 + set val [wappInt-enc-url $val] 732 + puts $chan "Set-Cookie: $nm=$val; HttpOnly; Path=/\r" 733 + } 734 + } 735 + } 736 + } 737 + if {[string match text/* $mimetype]} { 738 + set reply [encoding convertto utf-8 [dict get $wapp .reply]] 739 + if {[regexp {\ygzip\y} [wapp-param HTTP_ACCEPT_ENCODING]]} { 740 + catch { 741 + set x [zlib gzip $reply] 742 + set reply $x 743 + puts $chan "Content-Encoding: gzip\r" 744 + } 745 + } 746 + } else { 747 + set reply [dict get $wapp .reply] 748 + } 749 + puts $chan "Content-Length: [string length $reply]\r" 750 + puts $chan \r 751 + puts -nonewline $chan $reply 752 + flush $chan 753 + wappInt-close-channel $chan 754 +} 755 + 756 +# This routine runs just prior to request-handler dispatch. The 757 +# default implementation is a no-op, but applications can override 758 +# to do additional transformations or checks. 759 +# 760 +proc wapp-before-dispatch-hook {} {return} 761 + 762 +# Process a single CGI request 763 +# 764 +proc wappInt-handle-cgi-request {} { 765 + global wapp env 766 + foreach key { 767 + CONTENT_LENGTH 768 + CONTENT_TYPE 769 + DOCUMENT_ROOT 770 + HTTP_ACCEPT_ENCODING 771 + HTTP_COOKIE 772 + HTTP_HOST 773 + HTTP_REFERER 774 + HTTP_USER_AGENT 775 + HTTPS 776 + PATH_INFO 777 + QUERY_STRING 778 + REMOTE_ADDR 779 + REQUEST_METHOD 780 + REQUEST_URI 781 + REMOTE_USER 782 + SCRIPT_FILENAME 783 + SCRIPT_NAME 784 + SERVER_NAME 785 + SERVER_PORT 786 + SERVER_PROTOCOL 787 + } { 788 + if {[info exists env($key)]} { 789 + dict set wapp $key $env($key) 790 + } 791 + } 792 + set len 0 793 + if {[dict exists $wapp CONTENT_LENGTH]} { 794 + set len [dict get $wapp CONTENT_LENGTH] 795 + } 796 + if {$len>0} { 797 + fconfigure stdin -translation binary 798 + dict set wapp CONTENT [read stdin $len] 799 + } 800 + dict set wapp WAPP_MODE cgi 801 + fconfigure stdout -translation binary 802 + wappInt-handle-request stdout 1 803 +} 804 + 805 +# Process new text received on an inbound SCGI request 806 +# 807 +proc wappInt-scgi-readable {chan} { 808 + if {[catch [list wappInt-scgi-readable-unsafe $chan] msg]} { 809 + puts stderr "$msg\n$::errorInfo" 810 + wappInt-close-channel $chan 811 + } 812 +} 813 +proc wappInt-scgi-readable-unsafe {chan} { 814 + upvar #0 wappInt-$chan W wapp wapp 815 + if {![dict exists $W .toread]} { 816 + # If the .toread key is not set, that means we are still reading 817 + # the header. 818 + # 819 + # An SGI header is short. This implementation assumes the entire 820 + # header is available all at once. 821 + # 822 + dict set W .remove_addr [dict get $W REMOTE_ADDR] 823 + set req [read $chan 15] 824 + set n [string length $req] 825 + scan $req %d:%s len hdr 826 + incr len [string length "$len:,"] 827 + append hdr [read $chan [expr {$len-15}]] 828 + foreach {nm val} [split $hdr \000] { 829 + if {$nm==","} break 830 + dict set W $nm $val 831 + } 832 + set len 0 833 + if {[dict exists $W CONTENT_LENGTH]} { 834 + set len [dict get $W CONTENT_LENGTH] 835 + } 836 + if {$len>0} { 837 + # Still need to read the query content 838 + dict set W .toread $len 839 + } else { 840 + # There is no query content, so handle the request immediately 841 + dict set W SERVER_ADDR [dict get $W .remove_addr] 842 + set wapp $W 843 + wappInt-handle-request $chan 0 844 + } 845 + } else { 846 + # If .toread is set, that means we are reading the query content. 847 + # Continue reading until .toread reaches zero. 848 + set got [read $chan [dict get $W .toread]] 849 + dict append W CONTENT $got 850 + dict set W .toread [expr {[dict get $W .toread]-[string length $got]}] 851 + if {[dict get $W .toread]<=0} { 852 + # Handle the request as soon as all the query content is received 853 + dict set W SERVER_ADDR [dict get $W .remove_addr] 854 + set wapp $W 855 + wappInt-handle-request $chan 0 856 + } 857 + } 858 +} 859 + 860 +# Start up the wapp framework. Parameters are a list passed as the 861 +# single argument. 862 +# 863 +# -server $PORT Listen for HTTP requests on this TCP port $PORT 864 +# 865 +# -local $PORT Listen for HTTP requests on 127.0.0.1:$PORT 866 +# 867 +# -scgi $PORT Listen for SCGI requests on 127.0.0.1:$PORT 868 +# 869 +# -remote-scgi $PORT Listen for SCGI requests on TCP port $PORT 870 +# 871 +# -cgi Handle a single CGI request 872 +# 873 +# With no arguments, the behavior is called "auto". In "auto" mode, 874 +# if the GATEWAY_INTERFACE environment variable indicates CGI, then run 875 +# as CGI. Otherwise, start an HTTP server bound to the loopback address 876 +# only, on an arbitrary TCP port, and automatically launch a web browser 877 +# on that TCP port. 878 +# 879 +# Additional options: 880 +# 881 +# -fromip GLOB Reject any incoming request where the remote 882 +# IP address does not match the GLOB pattern. This 883 +# value defaults to '127.0.0.1' for -local and -scgi. 884 +# 885 +# -nowait Do not wait in the event loop. Return immediately 886 +# after all event handlers are established. 887 +# 888 +# -trace "puts" each request URL as it is handled, for 889 +# debugging 890 +# 891 +# -lint Run wapp-safety-check on the application instead 892 +# of running the application itself 893 +# 894 +# -Dvar=value Set TCL global variable "var" to "value" 895 +# 896 +# 897 +proc wapp-start {arglist} { 898 + global env 899 + set mode auto 900 + set port 0 901 + set nowait 0 902 + set fromip {} 903 + set n [llength $arglist] 904 + for {set i 0} {$i<$n} {incr i} { 905 + set term [lindex $arglist $i] 906 + if {[string match --* $term]} {set term [string range $term 1 end]} 907 + switch -glob -- $term { 908 + -server { 909 + incr i; 910 + set mode "server" 911 + set port [lindex $arglist $i] 912 + } 913 + -local { 914 + incr i; 915 + set mode "local" 916 + set fromip 127.0.0.1 917 + set port [lindex $arglist $i] 918 + } 919 + -scgi { 920 + incr i; 921 + set mode "scgi" 922 + set fromip 127.0.0.1 923 + set port [lindex $arglist $i] 924 + } 925 + -remote-scgi { 926 + incr i; 927 + set mode "remote-scgi" 928 + set port [lindex $arglist $i] 929 + } 930 + -cgi { 931 + set mode "cgi" 932 + } 933 + -fromip { 934 + incr i 935 + set fromip [lindex $arglist $i] 936 + } 937 + -nowait { 938 + set nowait 1 939 + } 940 + -trace { 941 + proc wappInt-trace {} { 942 + set q [wapp-param QUERY_STRING] 943 + set uri [wapp-param BASE_URL][wapp-param PATH_INFO] 944 + if {$q!=""} {append uri ?$q} 945 + puts $uri 946 + } 947 + } 948 + -lint { 949 + set res [wapp-safety-check] 950 + if {$res!=""} { 951 + puts "Potential problems in this code:" 952 + puts $res 953 + exit 1 954 + } else { 955 + exit 956 + } 957 + } 958 + -D*=* { 959 + if {[regexp {^.D([^=]+)=(.*)$} $term all var val]} { 960 + set ::$var $val 961 + } 962 + } 963 + default { 964 + error "unknown option: $term" 965 + } 966 + } 967 + } 968 + if {$mode=="auto"} { 969 + if {[info exists env(GATEWAY_INTERFACE)] 970 + && [string match CGI/1.* $env(GATEWAY_INTERFACE)]} { 971 + set mode cgi 972 + } else { 973 + set mode local 974 + } 975 + } 976 + if {$mode=="cgi"} { 977 + wappInt-handle-cgi-request 978 + } else { 979 + wappInt-start-listener $port $mode $fromip 980 + if {!$nowait} { 981 + vwait ::forever 982 + } 983 + } 984 +} 985 + 986 +# Call this version 1.0 987 +package provide wapp 1.0
Added test/wapptest.tcl.
1 +#!/bin/sh 2 +# \ 3 +exec wapptclsh "$0" ${1+"$@"} 4 + 5 +# package required wapp 6 +source [file join [file dirname [info script]] wapp.tcl] 7 + 8 +# Read the data from the releasetest_data.tcl script. 9 +# 10 +source [file join [file dirname [info script]] releasetest_data.tcl] 11 + 12 +# Variables set by the "control" form: 13 +# 14 +# G(platform) - User selected platform. 15 +# G(test) - Set to "Normal", "Veryquick", "Smoketest" or "Build-Only". 16 +# G(keep) - Boolean. True to delete no files after each test. 17 +# G(msvc) - Boolean. True to use MSVC as the compiler. 18 +# G(tcl) - Use Tcl from this directory for builds. 19 +# G(jobs) - How many sub-processes to run simultaneously. 20 +# 21 +set G(platform) $::tcl_platform(os)-$::tcl_platform(machine) 22 +set G(test) Normal 23 +set G(keep) 0 24 +set G(msvc) 0 25 +set G(tcl) [::tcl::pkgconfig get libdir,install] 26 +set G(jobs) 3 27 +set G(debug) 0 28 + 29 +proc wapptest_init {} { 30 + global G 31 + 32 + set lSave [list platform test keep msvc tcl jobs debug] 33 + foreach k $lSave { set A($k) $G($k) } 34 + array unset G 35 + foreach k $lSave { set G($k) $A($k) } 36 + 37 + # The root of the SQLite source tree. 38 + set G(srcdir) [file dirname [file dirname [info script]]] 39 + 40 + # releasetest.tcl script 41 + set G(releaseTest) [file join [file dirname [info script]] releasetest.tcl] 42 + 43 + set G(sqlite_version) "unknown" 44 + 45 + # Either "config", "running" or "stopped": 46 + set G(state) "config" 47 + 48 + set G(hostname) "(unknown host)" 49 + catch { set G(hostname) [exec hostname] } 50 + set G(host) $G(hostname) 51 + append G(host) " $::tcl_platform(os) $::tcl_platform(osVersion)" 52 + append G(host) " $::tcl_platform(machine) $::tcl_platform(byteOrder)" 53 +} 54 + 55 +# Check to see if there are uncommitted changes in the SQLite source 56 +# directory. Return true if there are, or false otherwise. 57 +# 58 +proc check_uncommitted {} { 59 + global G 60 + set ret 0 61 + set pwd [pwd] 62 + cd $G(srcdir) 63 + if {[catch {exec fossil changes} res]==0 && [string trim $res]!=""} { 64 + set ret 1 65 + } 66 + cd $pwd 67 + return $ret 68 +} 69 + 70 +proc generate_fossil_info {} { 71 + global G 72 + set pwd [pwd] 73 + cd $G(srcdir) 74 + if {[catch {exec fossil info} r1]} return 75 + if {[catch {exec fossil changes} r2]} return 76 + cd $pwd 77 + 78 + foreach line [split $r1 "\n"] { 79 + if {[regexp {^checkout: *(.*)$} $line -> co]} { 80 + wapp-trim { <br> %html($co) } 81 + } 82 + } 83 + 84 + if {[string trim $r2]!=""} { 85 + wapp-trim { 86 + <br><span class=warning> 87 + WARNING: Uncommitted changes in checkout 88 + </span> 89 + } 90 + } 91 +} 92 + 93 +# If the application is in "config" state, set the contents of the 94 +# ::G(test_array) global to reflect the tests that will be run. If the 95 +# app is in some other state ("running" or "stopped"), this command 96 +# is a no-op. 97 +# 98 +proc set_test_array {} { 99 + global G 100 + if { $G(state)=="config" } { 101 + set G(test_array) [list] 102 + foreach {config target} $::Platforms($G(platform)) { 103 + 104 + # If using MSVC, do not run sanitize or valgrind tests. Or the 105 + # checksymbols test. 106 + if {$G(msvc) && ( 107 + "Sanitize" == $config 108 + || "checksymbols" in $target 109 + || "valgrindtest" in $target 110 + )} { 111 + continue 112 + } 113 + 114 + # If the test mode is not "Normal", override the target. 115 + # 116 + if {$target!="checksymbols" && $G(platform)!="Failure-Detection"} { 117 + switch -- $G(test) { 118 + Veryquick { set target quicktest } 119 + Smoketest { set target smoketest } 120 + Build-Only { 121 + set target testfixture 122 + if {$::tcl_platform(platform)=="windows"} { 123 + set target testfixture.exe 124 + } 125 + } 126 + } 127 + } 128 + 129 + lappend G(test_array) [dict create config $config target $target] 130 + 131 + set exclude [list checksymbols valgrindtest fuzzoomtest] 132 + if {$G(debug) && !($target in $exclude)} { 133 + set debug_idx [lsearch -glob $::Configs($config) -DSQLITE_DEBUG*] 134 + set xtarget $target 135 + regsub -all {fulltest[a-z]*} $xtarget test xtarget 136 + if {$debug_idx<0} { 137 + lappend G(test_array) [ 138 + dict create config $config-(Debug) target $xtarget 139 + ] 140 + } else { 141 + lappend G(test_array) [ 142 + dict create config $config-(NDebug) target $xtarget 143 + ] 144 + } 145 + } 146 + } 147 + } 148 +} 149 + 150 +proc count_tests_and_errors {name logfile} { 151 + global G 152 + 153 + set fd [open $logfile rb] 154 + set seen 0 155 + while {![eof $fd]} { 156 + set line [gets $fd] 157 + if {[regexp {(\d+) errors out of (\d+) tests} $line all nerr ntest]} { 158 + incr G(test.$name.nError) $nerr 159 + incr G(test.$name.nTest) $ntest 160 + set seen 1 161 + if {$nerr>0} { 162 + set G(test.$name.errmsg) $line 163 + } 164 + } 165 + if {[regexp {runtime error: +(.*)} $line all msg]} { 166 + # skip over "value is outside range" errors 167 + if {[regexp {value .* is outside the range of representable} $line]} { 168 + # noop 169 + } else { 170 + incr G(test.$name.nError) 171 + if {$G(test.$name.errmsg)==""} { 172 + set G(test.$name.errmsg) $msg 173 + } 174 + } 175 + } 176 + if {[regexp {fatal error +(.*)} $line all msg]} { 177 + incr G(test.$name.nError) 178 + if {$G(test.$name.errmsg)==""} { 179 + set G(test.$name.errmsg) $msg 180 + } 181 + } 182 + if {[regexp {ERROR SUMMARY: (\d+) errors.*} $line all cnt] && $cnt>0} { 183 + incr G(test.$name.nError) 184 + if {$G(test.$name.errmsg)==""} { 185 + set G(test.$name.errmsg) $all 186 + } 187 + } 188 + if {[regexp {^VERSION: 3\.\d+.\d+} $line]} { 189 + set v [string range $line 9 end] 190 + if {$G(sqlite_version) eq "unknown"} { 191 + set G(sqlite_version) $v 192 + } elseif {$G(sqlite_version) ne $v} { 193 + set G(test.$name.errmsg) "version conflict: {$G(sqlite_version)} vs. {$v}" 194 + } 195 + } 196 + } 197 + close $fd 198 + if {$G(test) == "Build-Only"} { 199 + incr G(test.$name.nTest) 200 + if {$G(test.$name.nError)>0} { 201 + set errmsg "Build failed" 202 + } 203 + } elseif {!$seen} { 204 + set G(test.$name.errmsg) "Test did not complete" 205 + if {[file readable core]} { 206 + append G(test.$name.errmsg) " - core file exists" 207 + } 208 + } 209 +} 210 + 211 +proc slave_test_done {name rc} { 212 + global G 213 + set G(test.$name.done) [clock seconds] 214 + set G(test.$name.nError) 0 215 + set G(test.$name.nTest) 0 216 + set G(test.$name.errmsg) "" 217 + if {$rc} { 218 + incr G(test.$name.nError) 219 + } 220 + if {[file exists $G(test.$name.log)]} { 221 + count_tests_and_errors $name $G(test.$name.log) 222 + } 223 +} 224 + 225 +proc slave_fileevent {name} { 226 + global G 227 + set fd $G(test.$name.channel) 228 + 229 + if {[eof $fd]} { 230 + fconfigure $fd -blocking 1 231 + set rc [catch { close $fd }] 232 + unset G(test.$name.channel) 233 + slave_test_done $name $rc 234 + } else { 235 + set line [gets $fd] 236 + if {[string trim $line] != ""} { puts "Trace : $name - \"$line\"" } 237 + } 238 + 239 + do_some_stuff 240 +} 241 + 242 +proc do_some_stuff {} { 243 + global G 244 + 245 + # Count the number of running jobs. A running job has an entry named 246 + # "channel" in its dictionary. 247 + set nRunning 0 248 + set bFinished 1 249 + foreach j $G(test_array) { 250 + set name [dict get $j config] 251 + if { [info exists G(test.$name.channel)]} { incr nRunning } 252 + if {![info exists G(test.$name.done)]} { set bFinished 0 } 253 + } 254 + 255 + if {$bFinished} { 256 + set nError 0 257 + set nTest 0 258 + set nConfig 0 259 + foreach j $G(test_array) { 260 + set name [dict get $j config] 261 + incr nError $G(test.$name.nError) 262 + incr nTest $G(test.$name.nTest) 263 + incr nConfig 264 + } 265 + set G(result) "$nError errors from $nTest tests in $nConfig configurations." 266 + catch { 267 + append G(result) " SQLite version $G(sqlite_version)" 268 + } 269 + set G(state) "stopped" 270 + } else { 271 + set nLaunch [expr $G(jobs) - $nRunning] 272 + foreach j $G(test_array) { 273 + if {$nLaunch<=0} break 274 + set name [dict get $j config] 275 + if { ![info exists G(test.$name.channel)] 276 + && ![info exists G(test.$name.done)] 277 + } { 278 + set target [dict get $j target] 279 + set G(test.$name.start) [clock seconds] 280 + set fd [open "|[info nameofexecutable] $G(releaseTest) --slave" r+] 281 + set G(test.$name.channel) $fd 282 + fconfigure $fd -blocking 0 283 + fileevent $fd readable [list slave_fileevent $name] 284 + 285 + puts $fd [list 0 $G(msvc) 0 $G(keep)] 286 + 287 + set wtcl "" 288 + if {$G(tcl)!=""} { set wtcl "--with-tcl=$G(tcl)" } 289 + 290 + # If this configuration is named <name>-(Debug) or <name>-(NDebug), 291 + # then add or remove the SQLITE_DEBUG option from the base 292 + # configuration before running the test. 293 + if {[regexp -- {(.*)-(\(.*\))} $name -> head tail]} { 294 + set opts $::Configs($head) 295 + if {$tail=="(Debug)"} { 296 + append opts " -DSQLITE_DEBUG=1 -DSQLITE_EXTRA_IFNULLROW=1" 297 + } else { 298 + regsub { *-DSQLITE_MEMDEBUG[^ ]* *} $opts { } opts 299 + regsub { *-DSQLITE_DEBUG[^ ]* *} $opts { } opts 300 + } 301 + } else { 302 + set opts $::Configs($name) 303 + } 304 + 305 + set L [make_test_suite $G(msvc) $wtcl $name $target $opts] 306 + puts $fd $L 307 + flush $fd 308 + set G(test.$name.log) [file join [lindex $L 1] test.log] 309 + incr nLaunch -1 310 + } 311 + } 312 + } 313 +} 314 + 315 +proc generate_select_widget {label id lOpt opt} { 316 + wapp-trim { 317 + <label> %string($label) </label> 318 + <select id=%string($id) name=%string($id)> 319 + } 320 + foreach o $lOpt { 321 + set selected "" 322 + if {$o==$opt} { set selected " selected=1" } 323 + wapp-subst "<option $selected>$o</option>" 324 + } 325 + wapp-trim { </select> } 326 +} 327 + 328 +proc generate_main_page {{extra {}}} { 329 + global G 330 + set_test_array 331 + 332 + set hostname $G(hostname) 333 + wapp-trim { 334 + <html> 335 + <head> 336 + <title> %html($hostname): wapptest.tcl </title> 337 + <link rel="stylesheet" type="text/css" href="style.css"/> 338 + </head> 339 + <body> 340 + } 341 + 342 + set host $G(host) 343 + wapp-trim { 344 + <div class="border">%string($host) 345 + } 346 + generate_fossil_info 347 + wapp-trim { 348 + </div> 349 + <div class="border" id=controls> 350 + <form action="control" method="post" name="control"> 351 + } 352 + 353 + # Build the "platform" select widget. 354 + set lOpt [array names ::Platforms] 355 + generate_select_widget Platform control_platform $lOpt $G(platform) 356 + 357 + # Build the "test" select widget. 358 + set lOpt [list Normal Veryquick Smoketest Build-Only] 359 + generate_select_widget Test control_test $lOpt $G(test) 360 + 361 + # Build the "jobs" select widget. Options are 1 to 8. 362 + generate_select_widget Jobs control_jobs {1 2 3 4 5 6 7 8} $G(jobs) 363 + 364 + switch $G(state) { 365 + config { 366 + set txt "Run Tests!" 367 + set id control_run 368 + } 369 + running { 370 + set txt "STOP Tests!" 371 + set id control_stop 372 + } 373 + stopped { 374 + set txt "Reset!" 375 + set id control_reset 376 + } 377 + } 378 + wapp-trim { 379 + <div class=right> 380 + <input id=%string($id) name=%string($id) type=submit value="%string($txt)"> 381 + </input> 382 + </div> 383 + } 384 + 385 + wapp-trim { 386 + <br><br> 387 + <label> Tcl: </label> 388 + <input id="control_tcl" name="control_tcl"></input> 389 + <label> Keep files: </label> 390 + <input id="control_keep" name="control_keep" type=checkbox value=1> 391 + </input> 392 + <label> Use MSVC: </label> 393 + <input id="control_msvc" name="control_msvc" type=checkbox value=1> 394 + <label> Debug tests: </label> 395 + <input id="control_debug" name="control_debug" type=checkbox value=1> 396 + </input> 397 + } 398 + wapp-trim { 399 + </form> 400 + } 401 + wapp-trim { 402 + </div> 403 + <div id=tests> 404 + } 405 + wapp-page-tests 406 + 407 + set script "script/$G(state).js" 408 + wapp-trim { 409 + </div> 410 + <script src=%string($script)></script> 411 + </body> 412 + </html> 413 + } 414 +} 415 + 416 +proc wapp-default {} { 417 + generate_main_page 418 +} 419 + 420 +proc wapp-page-tests {} { 421 + global G 422 + wapp-trim { <table class="border" width=100%> } 423 + foreach t $G(test_array) { 424 + set config [dict get $t config] 425 + set target [dict get $t target] 426 + 427 + set class "testwait" 428 + set seconds "" 429 + 430 + if {[info exists G(test.$config.log)]} { 431 + if {[info exists G(test.$config.channel)]} { 432 + set class "testrunning" 433 + set seconds [expr [clock seconds] - $G(test.$config.start)] 434 + } elseif {[info exists G(test.$config.done)]} { 435 + if {$G(test.$config.nError)>0} { 436 + set class "testfail" 437 + } else { 438 + set class "testdone" 439 + } 440 + set seconds [expr $G(test.$config.done) - $G(test.$config.start)] 441 + } 442 + 443 + set min [format %.2d [expr ($seconds / 60) % 60]] 444 + set hr [format %.2d [expr $seconds / 3600]] 445 + set sec [format %.2d [expr $seconds % 60]] 446 + set seconds "$hr:$min:$sec" 447 + } 448 + 449 + wapp-trim { 450 + <tr class=%string($class)> 451 + <td class="nowrap"> %html($config) 452 + <td class="padleft nowrap"> %html($target) 453 + <td class="padleft nowrap"> %html($seconds) 454 + <td class="padleft nowrap"> 455 + } 456 + if {[info exists G(test.$config.log)]} { 457 + set log $G(test.$config.log) 458 + set uri "log/$log" 459 + wapp-trim { 460 + <a href=%url($uri)> %html($log) </a> 461 + } 462 + } 463 + if {[info exists G(test.$config.errmsg)] && $G(test.$config.errmsg)!=""} { 464 + set errmsg $G(test.$config.errmsg) 465 + wapp-trim { 466 + <tr class=testfail> 467 + <td> <td class="padleft" colspan=3> %html($errmsg) 468 + } 469 + } 470 + } 471 + 472 + wapp-trim { </table> } 473 + 474 + if {[info exists G(result)]} { 475 + set res $G(result) 476 + wapp-trim { 477 + <div class=border id=result> %string($res) </div> 478 + } 479 + } 480 +} 481 + 482 +# URI: /control 483 +# 484 +# Whenever the form at the top of the application page is submitted, it 485 +# is submitted here. 486 +# 487 +proc wapp-page-control {} { 488 + global G 489 + if {$::G(state)=="config"} { 490 + set lControls [list platform test tcl jobs keep msvc debug] 491 + set G(msvc) 0 492 + set G(keep) 0 493 + set G(debug) 0 494 + } else { 495 + set lControls [list jobs] 496 + } 497 + foreach v $lControls { 498 + if {[wapp-param-exists control_$v]} { 499 + set G($v) [wapp-param control_$v] 500 + } 501 + } 502 + 503 + if {[wapp-param-exists control_run]} { 504 + # This is a "run test" command. 505 + set_test_array 506 + set ::G(state) "running" 507 + } 508 + 509 + if {[wapp-param-exists control_stop]} { 510 + # A "STOP tests" command. 511 + set G(state) "stopped" 512 + set G(result) "Test halted by user" 513 + foreach j $G(test_array) { 514 + set name [dict get $j config] 515 + if { [info exists G(test.$name.channel)] } { 516 + close $G(test.$name.channel) 517 + unset G(test.$name.channel) 518 + slave_test_done $name 1 519 + } 520 + } 521 + } 522 + 523 + if {[wapp-param-exists control_reset]} { 524 + # A "reset app" command. 525 + set G(state) "config" 526 + wapptest_init 527 + } 528 + 529 + if {$::G(state) == "running"} { 530 + do_some_stuff 531 + } 532 + wapp-redirect / 533 +} 534 + 535 +# URI: /style.css 536 +# 537 +# Return the stylesheet for the application main page. 538 +# 539 +proc wapp-page-style.css {} { 540 + wapp-subst { 541 + 542 + /* The boxes with black borders use this class */ 543 + .border { 544 + border: 3px groove #444444; 545 + padding: 1em; 546 + margin-top: 1em; 547 + margin-bottom: 1em; 548 + } 549 + 550 + /* Float to the right (used for the Run/Stop/Reset button) */ 551 + .right { float: right; } 552 + 553 + /* Style for the large red warning at the top of the page */ 554 + .warning { 555 + color: red; 556 + font-weight: bold; 557 + } 558 + 559 + /* Styles used by cells in the test table */ 560 + .padleft { padding-left: 5ex; } 561 + .nowrap { white-space: nowrap; } 562 + 563 + /* Styles for individual tests, depending on the outcome */ 564 + .testwait { } 565 + .testrunning { color: blue } 566 + .testdone { color: green } 567 + .testfail { color: red } 568 + } 569 +} 570 + 571 +# URI: /script/${state}.js 572 +# 573 +# The last part of this URI is always "config.js", "running.js" or 574 +# "stopped.js", depending on the state of the application. It returns 575 +# the javascript part of the front-end for the requested state to the 576 +# browser. 577 +# 578 +proc wapp-page-script {} { 579 + regexp {[^/]*$} [wapp-param REQUEST_URI] script 580 + 581 + set tcl $::G(tcl) 582 + set keep $::G(keep) 583 + set msvc $::G(msvc) 584 + set debug $::G(debug) 585 + 586 + wapp-subst { 587 + var lElem = \["control_platform", "control_test", "control_msvc", 588 + "control_jobs", "control_debug" 589 + \]; 590 + lElem.forEach(function(e) { 591 + var elem = document.getElementById(e); 592 + elem.addEventListener("change", function() { control.submit() } ); 593 + }) 594 + 595 + elem = document.getElementById("control_tcl"); 596 + elem.value = "%string($tcl)" 597 + 598 + elem = document.getElementById("control_keep"); 599 + elem.checked = %string($keep); 600 + 601 + elem = document.getElementById("control_msvc"); 602 + elem.checked = %string($msvc); 603 + 604 + elem = document.getElementById("control_debug"); 605 + elem.checked = %string($debug); 606 + } 607 + 608 + if {$script != "config.js"} { 609 + wapp-subst { 610 + var lElem = \["control_platform", "control_test", 611 + "control_tcl", "control_keep", "control_msvc", 612 + "control_debug" 613 + \]; 614 + lElem.forEach(function(e) { 615 + var elem = document.getElementById(e); 616 + elem.disabled = true; 617 + }) 618 + } 619 + } 620 + 621 + if {$script == "running.js"} { 622 + wapp-subst { 623 + function reload_tests() { 624 + fetch('tests') 625 + .then( data => data.text() ) 626 + .then( data => { 627 + document.getElementById("tests").innerHTML = data; 628 + }) 629 + .then( data => { 630 + if( document.getElementById("result") ){ 631 + document.location = document.location; 632 + } else { 633 + setTimeout(reload_tests, 1000) 634 + } 635 + }); 636 + } 637 + 638 + setTimeout(reload_tests, 1000) 639 + } 640 + } 641 +} 642 + 643 +# URI: /env 644 +# 645 +# This is for debugging only. Serves no other purpose. 646 +# 647 +proc wapp-page-env {} { 648 + wapp-allow-xorigin-params 649 + wapp-trim { 650 + <h1>Wapp Environment</h1>\n<pre> 651 + <pre>%html([wapp-debug-env])</pre> 652 + } 653 +} 654 + 655 +# URI: /log/dirname/test.log 656 +# 657 +# This URI reads file "dirname/test.log" from disk, wraps it in a <pre> 658 +# block, and returns it to the browser. Use for viewing log files. 659 +# 660 +proc wapp-page-log {} { 661 + set log [string range [wapp-param REQUEST_URI] 5 end] 662 + set fd [open $log] 663 + set data [read $fd] 664 + close $fd 665 + wapp-trim { 666 + <pre> 667 + %html($data) 668 + </pre> 669 + } 670 +} 671 + 672 +wapptest_init 673 +wapp-start $argv 674 +
Changes to test/where.test.
1434 1434 CREATE TABLE t1(a INTEGER PRIMARY KEY); 1435 1435 INSERT INTO t1(a) VALUES(1),(2),(3); 1436 1436 CREATE TABLE t2(x INTEGER PRIMARY KEY, y INT); 1437 1437 INSERT INTO t2(y) VALUES(2),(3); 1438 1438 SELECT * FROM t1, t2 WHERE a=y AND y=3; 1439 1439 } {3 2 3} 1440 1440 1441 +#------------------------------------------------------------------------- 1442 +# 1443 +reset_db 1444 +do_execsql_test where-24.0 { 1445 + CREATE TABLE t1(a INTEGER PRIMARY KEY, b); 1446 + INSERT INTO t1 VALUES(1, 'one'); 1447 + INSERT INTO t1 VALUES(2, 'two'); 1448 + INSERT INTO t1 VALUES(3, 'three'); 1449 + INSERT INTO t1 VALUES(4, 'four'); 1450 +} 1451 + 1452 +foreach {tn sql res} { 1453 + 1 "SELECT b FROM t1" {one two three four} 1454 + 2 "SELECT b FROM t1 WHERE a<4" {one two three} 1455 + 3 "SELECT b FROM t1 WHERE a>1" {two three four} 1456 + 4 "SELECT b FROM t1 WHERE a>1 AND a<4" {two three} 1457 + 1458 + 5 "SELECT b FROM t1 WHERE a>? AND a<4" {} 1459 + 6 "SELECT b FROM t1 WHERE a>1 AND a<?" {} 1460 + 7 "SELECT b FROM t1 WHERE a>? AND a<?" {} 1461 + 1462 + 7 "SELECT b FROM t1 WHERE a>=? AND a<=4" {} 1463 + 8 "SELECT b FROM t1 WHERE a>=1 AND a<=?" {} 1464 + 9 "SELECT b FROM t1 WHERE a>=? AND a<=?" {} 1465 +} { 1466 + set rev [list] 1467 + foreach r $res { set rev [concat $r $rev] } 1468 + 1469 + do_execsql_test where-24.$tn.1 "$sql" $res 1470 + do_execsql_test where-24.$tn.2 "$sql ORDER BY rowid" $res 1471 + do_execsql_test where-24.$tn.3 "$sql ORDER BY rowid DESC" $rev 1472 + 1473 + do_execsql_test where-24-$tn.4 " 1474 + BEGIN; 1475 + DELETE FROM t1; 1476 + $sql; 1477 + $sql ORDER BY rowid; 1478 + $sql ORDER BY rowid DESC; 1479 + ROLLBACK; 1480 + " 1481 +} 1482 + 1483 +#------------------------------------------------------------------------- 1484 +# 1485 +reset_db 1486 +do_execsql_test where-25.0 { 1487 + CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); 1488 + CREATE UNIQUE INDEX i1 ON t1(c); 1489 + INSERT INTO t1 VALUES(1, 'one', 'i'); 1490 + INSERT INTO t1 VALUES(2, 'two', 'ii'); 1491 + 1492 + CREATE TABLE t2(a INTEGER PRIMARY KEY, b, c); 1493 + CREATE UNIQUE INDEX i2 ON t2(c); 1494 + INSERT INTO t2 VALUES(1, 'one', 'i'); 1495 + INSERT INTO t2 VALUES(2, 'two', 'ii'); 1496 + INSERT INTO t2 VALUES(3, 'three', 'iii'); 1497 + 1498 + PRAGMA writable_schema = 1; 1499 + UPDATE sqlite_master SET rootpage = ( 1500 + SELECT rootpage FROM sqlite_master WHERE name = 'i2' 1501 + ) WHERE name = 'i1'; 1502 +} 1503 +db close 1504 +sqlite3 db test.db 1505 +do_catchsql_test where-25.1 { 1506 + DELETE FROM t1 WHERE c='iii' 1507 +} {1 {database disk image is malformed}} 1508 +do_catchsql_test where-25.2 { 1509 + INSERT INTO t1 VALUES(4, 'four', 'iii') 1510 + ON CONFLICT(c) DO UPDATE SET b=NULL 1511 +} {1 {database disk image is malformed}} 1512 + 1513 +reset_db 1514 +do_execsql_test where-25.3 { 1515 + CREATE TABLE t1(a PRIMARY KEY, b, c) WITHOUT ROWID; 1516 + CREATE UNIQUE INDEX i1 ON t1(c); 1517 + INSERT INTO t1 VALUES(1, 'one', 'i'); 1518 + INSERT INTO t1 VALUES(2, 'two', 'ii'); 1519 + 1520 + CREATE TABLE t2(a INTEGER PRIMARY KEY, b, c); 1521 + CREATE UNIQUE INDEX i2 ON t2(c); 1522 + INSERT INTO t2 VALUES(1, 'one', 'i'); 1523 + INSERT INTO t2 VALUES(2, 'two', 'ii'); 1524 + INSERT INTO t2 VALUES(3, 'three', 'iii'); 1525 + 1526 + PRAGMA writable_schema = 1; 1527 + UPDATE sqlite_master SET rootpage = ( 1528 + SELECT rootpage FROM sqlite_master WHERE name = 'i2' 1529 + ) WHERE name = 'i1'; 1530 +} 1531 +db close 1532 +sqlite3 db test.db 1533 +do_catchsql_test where-25.4 { 1534 + SELECT * FROM t1 WHERE c='iii' 1535 +} {0 {}} 1536 +do_catchsql_test where-25.5 { 1537 + INSERT INTO t1 VALUES(4, 'four', 'iii') 1538 + ON CONFLICT(c) DO UPDATE SET b=NULL 1539 +} {1 {corrupt database}} 1540 + 1441 1541 finish_test 1542 +
Deleted tool/addopcodes.tcl.
1 -#!/usr/bin/tclsh 2 -# 3 -# This script appends additional token codes to the end of the 4 -# parse.h file that lemon generates. These extra token codes are 5 -# not used by the parser. But they are used by the tokenizer and/or 6 -# the code generator. 7 -# 8 -# 9 -set in [open [lindex $argv 0] rb] 10 -set max 0 11 -while {![eof $in]} { 12 - set line [gets $in] 13 - if {[regexp {^#define TK_} $line]} { 14 - puts $line 15 - set x [lindex $line 2] 16 - if {$x>$max} {set max $x} 17 - } 18 -} 19 -close $in 20 - 21 -# The following are the extra token codes to be added. SPACE and 22 -# ILLEGAL *must* be the last two token codes and they must be in that order. 23 -# 24 -set extras { 25 - TRUEFALSE 26 - ISNOT 27 - FUNCTION 28 - COLUMN 29 - AGG_FUNCTION 30 - AGG_COLUMN 31 - UMINUS 32 - UPLUS 33 - TRUTH 34 - REGISTER 35 - VECTOR 36 - SELECT_COLUMN 37 - IF_NULL_ROW 38 - ASTERISK 39 - SPAN 40 - END_OF_FILE 41 - UNCLOSED_STRING 42 - SPACE 43 - ILLEGAL 44 -} 45 -if {[lrange $extras end-1 end]!="SPACE ILLEGAL"} { 46 - error "SPACE and ILLEGAL must be the last two token codes and they\ 47 - must be in that order" 48 -} 49 -foreach x $extras { 50 - incr max 51 - puts [format "#define TK_%-29s %4d" $x $max] 52 -} 53 - 54 -# Some additional #defines related to token codes. 55 -# 56 -puts "\n/* The token codes above must all fit in 8 bits */" 57 -puts [format "#define %-20s %-6s" TKFLG_MASK 0xff] 58 -puts "\n/* Flags that can be added to a token code when it is not" 59 -puts "** being stored in a u8: */" 60 -foreach {fg val comment} { 61 - TKFLG_DONTFOLD 0x100 {/* Omit constant folding optimizations */} 62 -} { 63 - puts [format "#define %-20s %-6s %s" $fg $val $comment] 64 -}
Changes to tool/omittest.tcl.
188 188 SQLITE_OMIT_AUTOINIT \ 189 189 SQLITE_OMIT_AUTOMATIC_INDEX \ 190 190 SQLITE_OMIT_AUTORESET \ 191 191 SQLITE_OMIT_AUTOVACUUM \ 192 192 SQLITE_OMIT_BETWEEN_OPTIMIZATION \ 193 193 SQLITE_OMIT_BLOB_LITERAL \ 194 194 SQLITE_OMIT_BTREECOUNT \ 195 - SQLITE_OMIT_BUILTIN_TEST \ 196 195 SQLITE_OMIT_CAST \ 197 196 SQLITE_OMIT_CHECK \ 198 197 SQLITE_OMIT_COMPILEOPTION_DIAGS \ 199 198 SQLITE_OMIT_COMPLETE \ 200 199 SQLITE_OMIT_COMPOUND_SELECT \ 200 + SQLITE_OMIT_CONFLICT_CLAUSE \ 201 201 SQLITE_OMIT_CTE \ 202 202 SQLITE_OMIT_DATETIME_FUNCS \ 203 203 SQLITE_OMIT_DECLTYPE \ 204 204 SQLITE_OMIT_DEPRECATED \ 205 + SQLITE_OMIT_DISKIO \ 205 206 SQLITE_OMIT_EXPLAIN \ 206 207 SQLITE_OMIT_FLAG_PRAGMAS \ 207 208 SQLITE_OMIT_FLOATING_POINT \ 208 209 SQLITE_OMIT_FOREIGN_KEY \ 209 210 SQLITE_OMIT_GET_TABLE \ 211 + SQLITE_OMIT_HEX_INTEGER \ 210 212 SQLITE_OMIT_INCRBLOB \ 211 213 SQLITE_OMIT_INTEGRITY_CHECK \ 212 214 SQLITE_OMIT_LIKE_OPTIMIZATION \ 213 215 SQLITE_OMIT_LOAD_EXTENSION \ 214 216 SQLITE_OMIT_LOCALTIME \ 215 217 SQLITE_OMIT_LOOKASIDE \ 216 218 SQLITE_OMIT_MEMORYDB \ 219 + SQLITE_OMIT_MEMORY_ALLOCATION \ 217 220 SQLITE_OMIT_OR_OPTIMIZATION \ 218 221 SQLITE_OMIT_PAGER_PRAGMAS \ 222 + SQLITE_OMIT_PARSER_TRACE \ 223 + SQLITE_OMIT_POPEN \ 219 224 SQLITE_OMIT_PRAGMA \ 220 225 SQLITE_OMIT_PROGRESS_CALLBACK \ 221 226 SQLITE_OMIT_QUICKBALANCE \ 227 + SQLITE_OMIT_RANDOMNESS \ 222 228 SQLITE_OMIT_REINDEX \ 223 229 SQLITE_OMIT_SCHEMA_PRAGMAS \ 224 230 SQLITE_OMIT_SCHEMA_VERSION_PRAGMAS \ 225 231 SQLITE_OMIT_SHARED_CACHE \ 232 + SQLITE_OMIT_SHUTDOWN_DIRECTORIES \ 226 233 SQLITE_OMIT_SUBQUERY \ 227 234 SQLITE_OMIT_TCL_VARIABLE \ 228 235 SQLITE_OMIT_TEMPDB \ 236 + SQLITE_OMIT_TEST_CONTROL \ 229 237 SQLITE_OMIT_TRACE \ 230 238 SQLITE_OMIT_TRIGGER \ 231 239 SQLITE_OMIT_TRUNCATE_OPTIMIZATION \ 232 - SQLITE_OMIT_UNIQUE_ENFORCEMENT \ 240 + SQLITE_OMIT_UPSERT \ 233 241 SQLITE_OMIT_UTF16 \ 234 242 SQLITE_OMIT_VACUUM \ 235 243 SQLITE_OMIT_VIEW \ 236 244 SQLITE_OMIT_VIRTUALTABLE \ 237 245 SQLITE_OMIT_WAL \ 246 + SQLITE_OMIT_WINDOWFUNC \ 238 247 SQLITE_OMIT_WSD \ 239 248 SQLITE_OMIT_XFER_OPT \ 240 249 ] 241 250 242 251 set ::ENABLE_SYMBOLS [list \ 243 252 SQLITE_DISABLE_DIRSYNC \ 244 253 SQLITE_DISABLE_LFS \
Changes to tool/symbols.sh.
1 1 #!/bin/sh 2 2 # 3 3 # Run this script in a directory that contains a valid SQLite makefile in 4 4 # order to verify that unintentionally exported symbols. 5 5 # 6 6 make sqlite3.c 7 7 8 -echo '****** Exported symbols from a build including RTREE, FTS4 & ICU ******' 8 +echo '****** Exported symbols from a build including RTREE, FTS4 & FTS5 ******' 9 9 gcc -c -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_RTREE \ 10 10 -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_STAT3 \ 11 11 -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_ENABLE_UNLOCK_NOTIFY \ 12 12 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_ATOMIC_WRITE \ 13 - -DSQLITE_ENABLE_ICU -DSQLITE_ENABLE_PREUPDATE_HOOK -DSQLITE_ENABLE_SESSION \ 13 + -DSQLITE_ENABLE_PREUPDATE_HOOK -DSQLITE_ENABLE_SESSION \ 14 + -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_GEOPOLY \ 14 15 sqlite3.c 15 16 nm sqlite3.o | grep ' [TD] ' | sort -k 3 16 17 17 -echo '****** Surplus symbols from a build including RTREE, FTS4 & ICU ******' 18 -nm sqlite3.o | grep ' [TD] ' | grep -v ' .*sqlite3_' 18 +echo '****** Surplus symbols from a build including RTREE, FTS4 & FTS5 ******' 19 +nm sqlite3.o | grep ' [TD] ' | 20 + egrep -v ' .*sqlite3(session|rebaser|changeset|changegroup)?_' 19 21 20 22 echo '****** Dependencies of the core. No extensions. No OS interface *******' 21 23 gcc -c -DSQLITE_ENABLE_MEMORY_MANAGEMENT -DSQLITE_ENABLE_STAT3 \ 22 24 -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_ENABLE_UNLOCK_NOTIFY \ 23 25 -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_ATOMIC_WRITE \ 24 26 -DSQLITE_OS_OTHER -DSQLITE_THREADSAFE=0 \ 25 27 sqlite3.c