/ Check-in [53f2100a]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merge updates from trunk.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | zonefile
Files: files | file ages | folders
SHA3-256: 53f2100a296f64d206f1caf2d754820a7898bb33b31a6ab2169957d228d5d2a0
User & Date: mistachkin 2018-02-23 13:45:05
Context
2018-02-23
13:58
Avoid running a test case that requires zstd in non-SQLITE_HAVE_ZSTD builds. check-in: d716dff4 user: dan tags: zonefile
13:45
Merge updates from trunk. check-in: 53f2100a user: mistachkin tags: zonefile
13:38
Fix harmless compiler warnings in the zipfile extension seen with MSVC. check-in: 15c587cf user: mistachkin tags: trunk
2018-02-22
21:06
Add tests cases and fix some minor zonefile problems. check-in: f4d42162 user: dan tags: zonefile
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to Makefile.in.

   587    587   SHELL_OPT += -DSQLITE_ENABLE_DBPAGE_VTAB
   588    588   SHELL_OPT += -DSQLITE_ENABLE_DBSTAT_VTAB
   589    589   SHELL_OPT += -DSQLITE_ENABLE_OFFSET_SQL_FUNC
   590    590   SHELL_OPT += -DSQLITE_INTROSPECTION_PRAGMAS
   591    591   FUZZERSHELL_OPT = -DSQLITE_ENABLE_JSON1
   592    592   FUZZCHECK_OPT = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_OSS_FUZZ
   593    593   FUZZCHECK_OPT += -DSQLITE_MAX_MEMORY=50000000
          594  +FUZZCHECK_OPT += -DSQLITE_PRINTF_PRECISION_LIMIT=1000
   594    595   FUZZCHECK_SRC = $(TOP)/test/fuzzcheck.c $(TOP)/test/ossfuzz.c
   595    596   DBFUZZ_OPT = 
   596    597   
   597    598   # This is the default Makefile target.  The objects listed here
   598    599   # are what get build when you type just "make" with no arguments.
   599    600   #
   600    601   all:	sqlite3.h libsqlite3.la sqlite3$(TEXE) $(HAVE_TCL:1=libtclsqlite3.la)

Changes to Makefile.msc.

  1611   1611   !ENDIF
  1612   1612   
  1613   1613   # <<mark>>
  1614   1614   # Extra compiler options for various test tools.
  1615   1615   #
  1616   1616   MPTESTER_COMPILE_OPTS = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_FTS5
  1617   1617   FUZZERSHELL_COMPILE_OPTS = -DSQLITE_ENABLE_JSON1
  1618         -FUZZCHECK_COMPILE_OPTS = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_OSS_FUZZ -DSQLITE_MAX_MEMORY=50000000
         1618  +FUZZCHECK_COMPILE_OPTS = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_OSS_FUZZ -DSQLITE_MAX_MEMORY=50000000 -DSQLITE_PRINTF_PRECISION_LIMIT=1000
  1619   1619   FUZZCHECK_SRC = $(TOP)\test\fuzzcheck.c $(TOP)\test\ossfuzz.c
  1620   1620   OSSSHELL_SRC = $(TOP)\test\ossshell.c $(TOP)\test\ossfuzz.c
  1621   1621   DBFUZZ_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION
  1622   1622   KV_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_DIRECT_OVERFLOW_READ
  1623   1623   DBSELFTEST_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5
  1624   1624   ST_COMPILE_OPTS = -DSQLITE_THREADSAFE=0
  1625   1625   
................................................................................
  2248   2248   TESTFIXTURE_FLAGS = -DTCLSH_INIT_PROC=sqlite3TestInit -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1
  2249   2249   TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERVER=1 -DSQLITE_PRIVATE=""
  2250   2250   TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CORE $(NO_WARN)
  2251   2251   TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERIES_CONSTRAINT_VERIFY=1
  2252   2252   TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_DEFAULT_PAGE_SIZE=1024
  2253   2253   TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB
  2254   2254   TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB
         2255  +TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_JSON1
  2255   2256   TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) $(TEST_CCONV_OPTS)
  2256   2257   
  2257   2258   TESTFIXTURE_SRC0 = $(TESTEXT) $(TESTSRC2)
  2258   2259   TESTFIXTURE_SRC1 = $(TESTEXT) $(SQLITE3C)
  2259   2260   !IF $(USE_AMALGAMATION)==0
  2260   2261   TESTFIXTURE_SRC = $(TESTSRC) $(TOP)\src\tclsqlite.c $(TESTFIXTURE_SRC0)
  2261   2262   !ELSE

Changes to README.md.

    10     10   
    11     11   ## Obtaining The Code
    12     12   
    13     13   SQLite sources are managed using the
    14     14   [Fossil](https://www.fossil-scm.org/), a distributed version control system
    15     15   that was specifically designed to support SQLite development.
    16     16   If you do not want to use Fossil, you can download tarballs or ZIP
    17         -archives as follows:
           17  +archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows:
    18     18   
    19         -  *  Lastest trunk check-in:
    20         -     <https://www.sqlite.org/src/tarball/sqlite.tar.gz> or
    21         -     <https://www.sqlite.org/src/zip/sqlite.zip>.
           19  +  *  Lastest trunk check-in as
           20  +     [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz),
           21  +     [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip), or
           22  +     [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar).
    22     23   
    23         -  *  Latest release:
    24         -     <https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release> or
    25         -     <https://www.sqlite.org/src/zip/sqlite.zip?r=release>.
           24  +  *  Latest release as
           25  +     [Tarball](https://www.sqlite.org/src/tarball/sqlite.tar.gz?r=release),
           26  +     [ZIP-archive](https://www.sqlite.org/src/zip/sqlite.zip?r=release), or
           27  +     [SQLite-archive](https://www.sqlite.org/src/sqlar/sqlite.sqlar?r=release).
    26     28   
    27     29     *  For other check-ins, substitute an appropriate branch name or
    28     30        tag or hash prefix for "release" in the URLs of the previous
    29     31        bullet.  Or browse the [timeline](https://www.sqlite.org/src/timeline)
    30     32        to locate the check-in desired, click on its information page link,
    31     33        then click on the "Tarball" or "ZIP Archive" links on the information
    32     34        page.

Changes to ext/fts3/fts3_write.c.

  5525   5525     sqlite3_vtab *pVtab,            /* FTS3 vtab object */
  5526   5526     int nArg,                       /* Size of argument array */
  5527   5527     sqlite3_value **apVal,          /* Array of arguments */
  5528   5528     sqlite_int64 *pRowid            /* OUT: The affected (or effected) rowid */
  5529   5529   ){
  5530   5530     Fts3Table *p = (Fts3Table *)pVtab;
  5531   5531     int rc = SQLITE_OK;             /* Return Code */
  5532         -  int isRemove = 0;               /* True for an UPDATE or DELETE */
  5533   5532     u32 *aSzIns = 0;                /* Sizes of inserted documents */
  5534   5533     u32 *aSzDel = 0;                /* Sizes of deleted documents */
  5535   5534     int nChng = 0;                  /* Net change in number of documents */
  5536   5535     int bInsertDone = 0;
  5537   5536   
  5538   5537     /* At this point it must be known if the %_stat table exists or not.
  5539   5538     ** So bHasStat may not be 2.  */
................................................................................
  5623   5622       goto update_out;
  5624   5623     }
  5625   5624   
  5626   5625     /* If this is a DELETE or UPDATE operation, remove the old record. */
  5627   5626     if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){
  5628   5627       assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER );
  5629   5628       rc = fts3DeleteByRowid(p, apVal[0], &nChng, aSzDel);
  5630         -    isRemove = 1;
  5631   5629     }
  5632   5630     
  5633   5631     /* If this is an INSERT or UPDATE operation, insert the new record. */
  5634   5632     if( nArg>1 && rc==SQLITE_OK ){
  5635   5633       int iLangid = sqlite3_value_int(apVal[2 + p->nColumn + 2]);
  5636   5634       if( bInsertDone==0 ){
  5637   5635         rc = fts3InsertData(p, apVal, pRowid);
  5638   5636         if( rc==SQLITE_CONSTRAINT && p->zContentTbl==0 ){
  5639   5637           rc = FTS_CORRUPT_VTAB;
  5640   5638         }
  5641   5639       }
  5642         -    if( rc==SQLITE_OK && (!isRemove || *pRowid!=p->iPrevDocid ) ){
         5640  +    if( rc==SQLITE_OK ){
  5643   5641         rc = fts3PendingTermsDocid(p, 0, iLangid, *pRowid);
  5644   5642       }
  5645   5643       if( rc==SQLITE_OK ){
  5646   5644         assert( p->iPrevDocid==*pRowid );
  5647   5645         rc = fts3InsertTerms(p, iLangid, apVal, aSzIns);
  5648   5646       }
  5649   5647       if( p->bHasDocsize ){

Changes to ext/misc/spellfix.c.

    14     14   ** to search a large vocabulary for close matches.  See separate
    15     15   ** documentation (http://www.sqlite.org/spellfix1.html) for details.
    16     16   */
    17     17   #include "sqlite3ext.h"
    18     18   SQLITE_EXTENSION_INIT1
    19     19   
    20     20   #ifndef SQLITE_AMALGAMATION
           21  +# if !defined(NDEBUG) && !defined(SQLITE_DEBUG)
           22  +#  define NDEBUG 1
           23  +# endif
           24  +# if defined(NDEBUG) && defined(SQLITE_DEBUG)
           25  +#  undef NDEBUG
           26  +# endif
    21     27   # include <string.h>
    22     28   # include <stdio.h>
    23     29   # include <stdlib.h>
    24     30   # include <assert.h>
    25     31   # define ALWAYS(X)  1
    26     32   # define NEVER(X)   0
    27     33     typedef unsigned char u8;
................................................................................
   647    653     memset(p, 0, sizeof(*p));
   648    654   }
   649    655   static void editDist3ConfigDelete(void *pIn){
   650    656     EditDist3Config *p = (EditDist3Config*)pIn;
   651    657     editDist3ConfigClear(p);
   652    658     sqlite3_free(p);
   653    659   }
          660  +
          661  +/* Compare the FROM values of two EditDist3Cost objects, for sorting.
          662  +** Return negative, zero, or positive if the A is less than, equal to,
          663  +** or greater than B.
          664  +*/
          665  +static int editDist3CostCompare(EditDist3Cost *pA, EditDist3Cost *pB){
          666  +  int n = pA->nFrom;
          667  +  int rc;
          668  +  if( n>pB->nFrom ) n = pB->nFrom;
          669  +  rc = strncmp(pA->a, pB->a, n);
          670  +  if( rc==0 ) rc = pA->nFrom - pB->nFrom;
          671  +  return rc;
          672  +}
          673  +
          674  +/*
          675  +** Merge together two sorted lists of EditDist3Cost objects, in order
          676  +** of increasing FROM.
          677  +*/
          678  +static EditDist3Cost *editDist3CostMerge(
          679  +  EditDist3Cost *pA,
          680  +  EditDist3Cost *pB
          681  +){
          682  +  EditDist3Cost *pHead = 0;
          683  +  EditDist3Cost **ppTail = &pHead;
          684  +  EditDist3Cost *p;
          685  +  while( pA && pB ){
          686  +    if( editDist3CostCompare(pA,pB)<=0 ){
          687  +      p = pA;
          688  +      pA = pA->pNext;
          689  +    }else{
          690  +      p = pB;
          691  +      pB = pB->pNext;
          692  +    }
          693  +    *ppTail = p;
          694  +    ppTail =  &p->pNext;
          695  +  }
          696  +  if( pA ){
          697  +    *ppTail = pA;
          698  +  }else{
          699  +    *ppTail = pB;
          700  +  }
          701  +  return pHead;
          702  +}
          703  +
          704  +/*
          705  +** Sort a list of EditDist3Cost objects into order of increasing FROM
          706  +*/
          707  +static EditDist3Cost *editDist3CostSort(EditDist3Cost *pList){
          708  +  EditDist3Cost *ap[60], *p;
          709  +  int i;
          710  +  int mx = 0;
          711  +  ap[0] = 0;
          712  +  ap[1] = 0;
          713  +  while( pList ){
          714  +    p = pList;
          715  +    pList = p->pNext;
          716  +    p->pNext = 0;
          717  +    for(i=0; ap[i]; i++){
          718  +      p = editDist3CostMerge(ap[i],p);
          719  +      ap[i] = 0;
          720  +    }
          721  +    ap[i] = p;
          722  +    if( i>mx ){
          723  +      mx = i;
          724  +      ap[i+1] = 0;
          725  +    }
          726  +  }
          727  +  p = 0;
          728  +  for(i=0; i<=mx; i++){
          729  +    if( ap[i] ) p = editDist3CostMerge(p,ap[i]);
          730  +  }
          731  +  return p;
          732  +}
   654    733   
   655    734   /*
   656    735   ** Load all edit-distance weights from a table.
   657    736   */
   658    737   static int editDist3ConfigLoad(
   659    738     EditDist3Config *p,      /* The edit distance configuration to load */
   660    739     sqlite3 *db,            /* Load from this database */
................................................................................
   681    760       int nTo = zTo ? sqlite3_column_bytes(pStmt, 2) : 0;
   682    761       int iCost = sqlite3_column_int(pStmt, 3);
   683    762   
   684    763       assert( zFrom!=0 || nFrom==0 );
   685    764       assert( zTo!=0 || nTo==0 );
   686    765       if( nFrom>100 || nTo>100 ) continue;
   687    766       if( iCost<0 ) continue;
          767  +    if( iCost>10000 ) continue;   /* Costs above 10K are considered infinite */
   688    768       if( pLang==0 || iLang!=iLangPrev ){
   689    769         EditDist3Lang *pNew;
   690    770         pNew = sqlite3_realloc64(p->a, (p->nLang+1)*sizeof(p->a[0]));
   691    771         if( pNew==0 ){ rc = SQLITE_NOMEM; break; }
   692    772         p->a = pNew;
   693    773         pLang = &p->a[p->nLang];
   694    774         p->nLang++;
................................................................................
   718    798         memcpy(pCost->a + nFrom, zTo, nTo);
   719    799         pCost->pNext = pLang->pCost;
   720    800         pLang->pCost = pCost; 
   721    801       }
   722    802     }
   723    803     rc2 = sqlite3_finalize(pStmt);
   724    804     if( rc==SQLITE_OK ) rc = rc2;
          805  +  if( rc==SQLITE_OK ){
          806  +    int iLang;
          807  +    for(iLang=0; iLang<p->nLang; iLang++){
          808  +      p->a[iLang].pCost = editDist3CostSort(p->a[iLang].pCost);
          809  +    }
          810  +  }
   725    811     return rc;
   726    812   }
   727    813   
   728    814   /*
   729    815   ** Return the length (in bytes) of a utf-8 character.  Or return a maximum
   730    816   ** of N.
   731    817   */
................................................................................
   745    831   }
   746    832   
   747    833   /*
   748    834   ** Return TRUE (non-zero) if the To side of the given cost matches
   749    835   ** the given string.
   750    836   */
   751    837   static int matchTo(EditDist3Cost *p, const char *z, int n){
          838  +  if( p->a[p->nFrom]!=z[0] ) return 0;
   752    839     if( p->nTo>n ) return 0;
   753    840     if( strncmp(p->a+p->nFrom, z, p->nTo)!=0 ) return 0;
   754    841     return 1;
   755    842   }
   756    843   
   757    844   /*
   758    845   ** Return TRUE (non-zero) if the From side of the given cost matches
   759    846   ** the given string.
   760    847   */
   761    848   static int matchFrom(EditDist3Cost *p, const char *z, int n){
   762    849     assert( p->nFrom<=n );
          850  +  if( p->a[0]!=z[0] ) return 0;
   763    851     if( strncmp(p->a, z, p->nFrom)!=0 ) return 0;
   764    852     return 1;
   765    853   }
   766    854   
   767    855   /*
   768    856   ** Return TRUE (non-zero) of the next FROM character and the next TO
   769    857   ** character are the same.
................................................................................
   772    860     EditDist3FromString *pStr,  /* Left hand string */
   773    861     int n1,                     /* Index of comparison character on the left */
   774    862     const char *z2,             /* Right-handl comparison character */
   775    863     int n2                      /* Bytes remaining in z2[] */
   776    864   ){
   777    865     int b1 = pStr->a[n1].nByte;
   778    866     if( b1>n2 ) return 0;
   779         -  if( memcmp(pStr->z+n1, z2, b1)!=0 ) return 0;
          867  +  if( pStr->z[n1]!=z2[0] ) return 0;
          868  +  if( strncmp(pStr->z+n1, z2, b1)!=0 ) return 0;
   780    869     return 1;
   781    870   }
   782    871   
   783    872   /*
   784    873   ** Delete an EditDist3FromString objecct
   785    874   */
   786    875   static void editDist3FromStringDelete(EditDist3FromString *p){
................................................................................
   854    943     }
   855    944     return pStr;
   856    945   }
   857    946   
   858    947   /*
   859    948   ** Update entry m[i] such that it is the minimum of its current value
   860    949   ** and m[j]+iCost.
   861         -**
   862         -** If the iCost is 1,000,000 or greater, then consider the cost to be
   863         -** infinite and skip the update.
   864    950   */
   865    951   static void updateCost(
   866    952     unsigned int *m,
   867    953     int i,
   868    954     int j,
   869    955     int iCost
   870    956   ){
          957  +  unsigned int b;
   871    958     assert( iCost>=0 );
   872         -  if( iCost<10000 ){
   873         -    unsigned int b = m[j] + iCost;
   874         -    if( b<m[i] ) m[i] = b;
   875         -  }
          959  +  assert( iCost<10000 );
          960  +  b = m[j] + iCost;
          961  +  if( b<m[i] ) m[i] = b;
   876    962   }
   877    963   
   878    964   /*
   879    965   ** How much stack space (int bytes) to use for Wagner matrix in 
   880    966   ** editDist3Core().  If more space than this is required, the entire
   881    967   ** matrix is taken from the heap.  To reduce the load on the memory
   882    968   ** allocator, make this value as large as practical for the
................................................................................
   932   1018     memset(a2, 0, sizeof(a2[0])*n2);
   933   1019   
   934   1020     /* Fill in the a1[] matrix for all characters of the TO string */
   935   1021     for(i2=0; i2<n2; i2++){
   936   1022       a2[i2].nByte = utf8Len((unsigned char)z2[i2], n2-i2);
   937   1023       for(p=pLang->pCost; p; p=p->pNext){
   938   1024         EditDist3Cost **apNew;
   939         -      if( p->nFrom>0 ) continue;
         1025  +      if( p->nFrom>0 ) break;
   940   1026         if( i2+p->nTo>n2 ) continue;
         1027  +      if( p->a[0]>z2[i2] ) break;
   941   1028         if( matchTo(p, z2+i2, n2-i2)==0 ) continue;
   942   1029         a2[i2].nIns++;
   943   1030         apNew = sqlite3_realloc64(a2[i2].apIns, sizeof(*apNew)*a2[i2].nIns);
   944   1031         if( apNew==0 ){
   945   1032           res = -1;  /* Out of memory */
   946   1033           goto editDist3Abort;
   947   1034         }
................................................................................
  2488   2575     if( zPattern==0 ){
  2489   2576       x.rc = SQLITE_NOMEM;
  2490   2577       goto filter_exit;
  2491   2578     }
  2492   2579     nPattern = (int)strlen(zPattern);
  2493   2580     if( zPattern[nPattern-1]=='*' ) nPattern--;
  2494   2581     zSql = sqlite3_mprintf(
  2495         -     "SELECT id, word, rank, k1"
         2582  +     "SELECT id, word, rank, coalesce(k1,word)"
  2496   2583        "  FROM \"%w\".\"%w_vocab\""
  2497   2584        " WHERE langid=%d AND k2>=?1 AND k2<?2",
  2498   2585        p->zDbName, p->zTableName, iLang
  2499   2586     );
  2500   2587     if( zSql==0 ){
  2501   2588       x.rc = SQLITE_NOMEM;
  2502   2589       pStmt = 0;
................................................................................
  2822   2909         sqlite3_free(zK1);
  2823   2910         return SQLITE_NOMEM;
  2824   2911       }
  2825   2912       if( sqlite3_value_type(argv[0])==SQLITE_NULL ){
  2826   2913         if( sqlite3_value_type(argv[1])==SQLITE_NULL ){
  2827   2914           spellfix1DbExec(&rc, db,
  2828   2915                  "INSERT INTO \"%w\".\"%w_vocab\"(rank,langid,word,k1,k2) "
  2829         -               "VALUES(%d,%d,%Q,%Q,%Q)",
         2916  +               "VALUES(%d,%d,%Q,nullif(%Q,%Q),%Q)",
  2830   2917                  p->zDbName, p->zTableName,
  2831         -               iRank, iLang, zWord, zK1, zK2
         2918  +               iRank, iLang, zWord, zK1, zWord, zK2
  2832   2919           );
  2833   2920         }else{
  2834   2921           newRowid = sqlite3_value_int64(argv[1]);
  2835   2922           spellfix1DbExec(&rc, db,
  2836   2923               "INSERT OR %s INTO \"%w\".\"%w_vocab\"(id,rank,langid,word,k1,k2) "
  2837         -            "VALUES(%lld,%d,%d,%Q,%Q,%Q)",
         2924  +            "VALUES(%lld,%d,%d,%Q,nullif(%Q,%Q),%Q)",
  2838   2925               zConflict, p->zDbName, p->zTableName,
  2839         -            newRowid, iRank, iLang, zWord, zK1, zK2
         2926  +            newRowid, iRank, iLang, zWord, zK1, zWord, zK2
  2840   2927           );
  2841   2928         }
  2842   2929         *pRowid = sqlite3_last_insert_rowid(db);
  2843   2930       }else{
  2844   2931         rowid = sqlite3_value_int64(argv[0]);
  2845   2932         newRowid = *pRowid = sqlite3_value_int64(argv[1]);
  2846   2933         spellfix1DbExec(&rc, db,
  2847   2934                "UPDATE OR %s \"%w\".\"%w_vocab\" SET id=%lld, rank=%d, langid=%d,"
  2848         -             " word=%Q, k1=%Q, k2=%Q WHERE id=%lld",
         2935  +             " word=%Q, k1=nullif(%Q,%Q), k2=%Q WHERE id=%lld",
  2849   2936                zConflict, p->zDbName, p->zTableName, newRowid, iRank, iLang,
  2850         -             zWord, zK1, zK2, rowid
         2937  +             zWord, zK1, zWord, zK2, rowid
  2851   2938         );
  2852   2939       }
  2853   2940       sqlite3_free(zK1);
  2854   2941       sqlite3_free(zK2);
  2855   2942     }
  2856   2943     return rc;
  2857   2944   }

Changes to ext/misc/zipfile.c.

   733    733     i64 JD = (i64)2440588 + mUnixTime / (24*60*60);
   734    734   
   735    735     int A, B, C, D, E;
   736    736     int yr, mon, day;
   737    737     int hr, min, sec;
   738    738   
   739    739     A = (int)((JD - 1867216.25)/36524.25);
   740         -  A = JD + 1 + A - (A/4);
          740  +  A = (int)(JD + 1 + A - (A/4));
   741    741     B = A + 1524;
   742    742     C = (int)((B - 122.1)/365.25);
   743    743     D = (36525*(C&32767))/100;
   744    744     E = (int)((B-D)/30.6001);
   745    745   
   746    746     day = B - D - (int)(30.6001*E);
   747    747     mon = (E<14 ? E-1 : E-13);
................................................................................
  1619   1619           pNew->cds.iCompression = (u16)iMethod;
  1620   1620           zipfileMtimeToDos(&pNew->cds, mTime);
  1621   1621           pNew->cds.crc32 = iCrc32;
  1622   1622           pNew->cds.szCompressed = nData;
  1623   1623           pNew->cds.szUncompressed = (u32)sz;
  1624   1624           pNew->cds.iExternalAttr = (mode<<16);
  1625   1625           pNew->cds.iOffset = (u32)pTab->szCurrent;
  1626         -        pNew->cds.nFile = nPath;
         1626  +        pNew->cds.nFile = (u16)nPath;
  1627   1627           pNew->mUnixTime = (u32)mTime;
  1628   1628           rc = zipfileAppendEntry(pTab, pNew, pData, nData);
  1629   1629           zipfileAddEntry(pTab, pOld, pNew);
  1630   1630         }
  1631   1631       }
  1632   1632     }
  1633   1633   
................................................................................
  1922   1922       rc = SQLITE_ERROR;
  1923   1923       goto zipfile_step_out;
  1924   1924     }
  1925   1925   
  1926   1926     /* Inspect the 'method' parameter. This must be either 0 (store), 8 (use
  1927   1927     ** deflate compression) or NULL (choose automatically).  */
  1928   1928     if( pMethod && SQLITE_NULL!=sqlite3_value_type(pMethod) ){
  1929         -    iMethod = sqlite3_value_int64(pMethod);
         1929  +    iMethod = (int)sqlite3_value_int64(pMethod);
  1930   1930       if( iMethod!=0 && iMethod!=8 ){
  1931   1931         zErr = sqlite3_mprintf("illegal method value: %d", iMethod);
  1932   1932         rc = SQLITE_ERROR;
  1933   1933         goto zipfile_step_out;
  1934   1934       }
  1935   1935     }
  1936   1936   
................................................................................
  1989   1989       }
  1990   1990     }
  1991   1991   
  1992   1992     /* Assemble the ZipfileEntry object for the new zip archive entry */
  1993   1993     e.cds.iVersionMadeBy = ZIPFILE_NEWENTRY_MADEBY;
  1994   1994     e.cds.iVersionExtract = ZIPFILE_NEWENTRY_REQUIRED;
  1995   1995     e.cds.flags = ZIPFILE_NEWENTRY_FLAGS;
  1996         -  e.cds.iCompression = iMethod;
         1996  +  e.cds.iCompression = (u16)iMethod;
  1997   1997     zipfileMtimeToDos(&e.cds, (u32)e.mUnixTime);
  1998   1998     e.cds.crc32 = iCrc32;
  1999   1999     e.cds.szCompressed = nData;
  2000   2000     e.cds.szUncompressed = szUncompressed;
  2001   2001     e.cds.iExternalAttr = (mode<<16);
  2002   2002     e.cds.iOffset = p->body.n;
  2003         -  e.cds.nFile = nName;
         2003  +  e.cds.nFile = (u16)nName;
  2004   2004     e.cds.zFile = zName;
  2005   2005   
  2006   2006     /* Append the LFH to the body of the new archive */
  2007   2007     nByte = ZIPFILE_LFH_FIXED_SZ + e.cds.nFile + 9;
  2008   2008     if( (rc = zipfileBufferGrow(&p->body, nByte)) ) goto zipfile_step_out;
  2009   2009     p->body.n += zipfileSerializeLFH(&e, &p->body.a[p->body.n]);
  2010   2010   
................................................................................
  2043   2043     int nZip;
  2044   2044     u8 *aZip;
  2045   2045   
  2046   2046     p = (ZipfileCtx*)sqlite3_aggregate_context(pCtx, sizeof(ZipfileCtx));
  2047   2047     if( p==0 ) return;
  2048   2048     if( p->nEntry>0 ){
  2049   2049       memset(&eocd, 0, sizeof(eocd));
  2050         -    eocd.nEntry = p->nEntry;
  2051         -    eocd.nEntryTotal = p->nEntry;
         2050  +    eocd.nEntry = (u16)p->nEntry;
         2051  +    eocd.nEntryTotal = (u16)p->nEntry;
  2052   2052       eocd.nSize = p->cds.n;
  2053   2053       eocd.iOffset = p->body.n;
  2054   2054   
  2055   2055       nZip = p->body.n + p->cds.n + ZIPFILE_EOCD_FIXED_SZ;
  2056   2056       aZip = (u8*)sqlite3_malloc(nZip);
  2057   2057       if( aZip==0 ){
  2058   2058         sqlite3_result_error_nomem(pCtx);

Changes to main.mk.

   516    516   SHELL_OPT += -DSQLITE_ENABLE_DBPAGE_VTAB
   517    517   SHELL_OPT += -DSQLITE_ENABLE_DBSTAT_VTAB
   518    518   SHELL_OPT += -DSQLITE_ENABLE_OFFSET_SQL_FUNC
   519    519   SHELL_OPT += -DSQLITE_INTROSPECTION_PRAGMAS
   520    520   FUZZERSHELL_OPT = -DSQLITE_ENABLE_JSON1
   521    521   FUZZCHECK_OPT = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_MEMSYS5
   522    522   FUZZCHECK_OPT += -DSQLITE_MAX_MEMORY=50000000
          523  +FUZZCHECK_OPT += -DSQLITE_PRINTF_PRECISION_LIMIT=1000
   523    524   DBFUZZ_OPT =
   524    525   KV_OPT = -DSQLITE_THREADSAFE=0 -DSQLITE_DIRECT_OVERFLOW_READ
   525    526   ST_OPT = -DSQLITE_THREADSAFE=0
   526    527   
   527    528   # This is the default Makefile target.  The objects listed here
   528    529   # are what get build when you type just "make" with no arguments.
   529    530   #

Changes to src/attach.c.

   499    499         return 1;
   500    500       }
   501    501       if( sqlite3FixExprList(pFix, pSelect->pOrderBy) ){
   502    502         return 1;
   503    503       }
   504    504       if( sqlite3FixExpr(pFix, pSelect->pLimit) ){
   505    505         return 1;
          506  +    }
          507  +    if( pSelect->pWith ){
          508  +      int i;
          509  +      for(i=0; i<pSelect->pWith->nCte; i++){
          510  +        if( sqlite3FixSelect(pFix, pSelect->pWith->a[i].pSelect) ){
          511  +          return 1;
          512  +        }
          513  +      }
   506    514       }
   507    515       pSelect = pSelect->pPrior;
   508    516     }
   509    517     return 0;
   510    518   }
   511    519   int sqlite3FixExpr(
   512    520     DbFixer *pFix,     /* Context of the fixation */

Changes to src/btree.c.

  4384   4384   ** BtCursor.info structure.  If it is not already valid, call
  4385   4385   ** btreeParseCell() to fill it in.
  4386   4386   **
  4387   4387   ** BtCursor.info is a cache of the information in the current cell.
  4388   4388   ** Using this cache reduces the number of calls to btreeParseCell().
  4389   4389   */
  4390   4390   #ifndef NDEBUG
         4391  +  static int cellInfoEqual(CellInfo *a, CellInfo *b){
         4392  +    if( a->nKey!=b->nKey ) return 0;
         4393  +    if( a->pPayload!=b->pPayload ) return 0;
         4394  +    if( a->nPayload!=b->nPayload ) return 0;
         4395  +    if( a->nLocal!=b->nLocal ) return 0;
         4396  +    if( a->nSize!=b->nSize ) return 0;
         4397  +    return 1;
         4398  +  }
  4391   4399     static void assertCellInfo(BtCursor *pCur){
  4392   4400       CellInfo info;
  4393   4401       memset(&info, 0, sizeof(info));
  4394   4402       btreeParseCell(pCur->pPage, pCur->ix, &info);
  4395         -    assert( CORRUPT_DB || memcmp(&info, &pCur->info, sizeof(info))==0 );
         4403  +    assert( CORRUPT_DB || cellInfoEqual(&info, &pCur->info) );
  4396   4404     }
  4397   4405   #else
  4398   4406     #define assertCellInfo(x)
  4399   4407   #endif
  4400   4408   static SQLITE_NOINLINE void getCellInfo(BtCursor *pCur){
  4401   4409     if( pCur->info.nSize==0 ){
  4402   4410       pCur->curFlags |= BTCF_ValidNKey;

Changes to src/global.c.

   254    254   ** Constant tokens for values 0 and 1.
   255    255   */
   256    256   const Token sqlite3IntTokens[] = {
   257    257      { "0", 1 },
   258    258      { "1", 1 }
   259    259   };
   260    260   
          261  +#ifdef VDBE_PROFILE
          262  +/*
          263  +** The following performance counter can be used in place of
          264  +** sqlite3Hwtime() for profiling.  This is a no-op on standard builds.
          265  +*/
          266  +sqlite3_uint64 sqlite3NProfileCnt = 0;
          267  +#endif
   261    268   
   262    269   /*
   263    270   ** The value of the "pending" byte must be 0x40000000 (1 byte past the
   264    271   ** 1-gibabyte boundary) in a compatible database.  SQLite never uses
   265    272   ** the database page that contains the pending byte.  It never attempts
   266    273   ** to read or write that page.  The pending byte page is set aside
   267    274   ** for use by the VFS layers as space for managing file locks.

Changes to src/mutex_unix.c.

    46     46   #if SQLITE_MUTEX_NREF
    47     47     volatile int nRef;         /* Number of entrances */
    48     48     volatile pthread_t owner;  /* Thread that is within this mutex */
    49     49     int trace;                 /* True to trace changes */
    50     50   #endif
    51     51   };
    52     52   #if SQLITE_MUTEX_NREF
    53         -#define SQLITE3_MUTEX_INITIALIZER {PTHREAD_MUTEX_INITIALIZER,0,0,(pthread_t)0,0}
           53  +# define SQLITE3_MUTEX_INITIALIZER(id) \
           54  +     {PTHREAD_MUTEX_INITIALIZER,id,0,(pthread_t)0,0}
    54     55   #elif defined(SQLITE_ENABLE_API_ARMOR)
    55         -#define SQLITE3_MUTEX_INITIALIZER { PTHREAD_MUTEX_INITIALIZER, 0 }
           56  +# define SQLITE3_MUTEX_INITIALIZER(id) { PTHREAD_MUTEX_INITIALIZER, id }
    56     57   #else
    57         -#define SQLITE3_MUTEX_INITIALIZER { PTHREAD_MUTEX_INITIALIZER }
           58  +#define SQLITE3_MUTEX_INITIALIZER(id) { PTHREAD_MUTEX_INITIALIZER }
    58     59   #endif
    59     60   
    60     61   /*
    61     62   ** The sqlite3_mutex_held() and sqlite3_mutex_notheld() routine are
    62     63   ** intended for use only inside assert() statements.  On some platforms,
    63     64   ** there might be race conditions that can cause these routines to
    64     65   ** deliver incorrect results.  In particular, if pthread_equal() is
................................................................................
   147    148   ** or SQLITE_MUTEX_RECURSIVE) is used then sqlite3_mutex_alloc()
   148    149   ** returns a different mutex on every call.  But for the static 
   149    150   ** mutex types, the same mutex is returned on every call that has
   150    151   ** the same type number.
   151    152   */
   152    153   static sqlite3_mutex *pthreadMutexAlloc(int iType){
   153    154     static sqlite3_mutex staticMutexes[] = {
   154         -    SQLITE3_MUTEX_INITIALIZER,
   155         -    SQLITE3_MUTEX_INITIALIZER,
   156         -    SQLITE3_MUTEX_INITIALIZER,
   157         -    SQLITE3_MUTEX_INITIALIZER,
   158         -    SQLITE3_MUTEX_INITIALIZER,
   159         -    SQLITE3_MUTEX_INITIALIZER,
   160         -    SQLITE3_MUTEX_INITIALIZER,
   161         -    SQLITE3_MUTEX_INITIALIZER,
   162         -    SQLITE3_MUTEX_INITIALIZER,
   163         -    SQLITE3_MUTEX_INITIALIZER,
   164         -    SQLITE3_MUTEX_INITIALIZER,
   165         -    SQLITE3_MUTEX_INITIALIZER
          155  +    SQLITE3_MUTEX_INITIALIZER(2),
          156  +    SQLITE3_MUTEX_INITIALIZER(3),
          157  +    SQLITE3_MUTEX_INITIALIZER(4),
          158  +    SQLITE3_MUTEX_INITIALIZER(5),
          159  +    SQLITE3_MUTEX_INITIALIZER(6),
          160  +    SQLITE3_MUTEX_INITIALIZER(7),
          161  +    SQLITE3_MUTEX_INITIALIZER(8),
          162  +    SQLITE3_MUTEX_INITIALIZER(9),
          163  +    SQLITE3_MUTEX_INITIALIZER(10),
          164  +    SQLITE3_MUTEX_INITIALIZER(11),
          165  +    SQLITE3_MUTEX_INITIALIZER(12),
          166  +    SQLITE3_MUTEX_INITIALIZER(13)
   166    167     };
   167    168     sqlite3_mutex *p;
   168    169     switch( iType ){
   169    170       case SQLITE_MUTEX_RECURSIVE: {
   170    171         p = sqlite3MallocZero( sizeof(*p) );
   171    172         if( p ){
   172    173   #ifdef SQLITE_HOMEGROWN_RECURSIVE_MUTEX
................................................................................
   177    178           /* Use a recursive mutex if it is available */
   178    179           pthread_mutexattr_t recursiveAttr;
   179    180           pthread_mutexattr_init(&recursiveAttr);
   180    181           pthread_mutexattr_settype(&recursiveAttr, PTHREAD_MUTEX_RECURSIVE);
   181    182           pthread_mutex_init(&p->mutex, &recursiveAttr);
   182    183           pthread_mutexattr_destroy(&recursiveAttr);
   183    184   #endif
          185  +#if SQLITE_MUTEX_NREF || defined(SQLITE_ENABLE_API_ARMOR)
          186  +        p->id = SQLITE_MUTEX_RECURSIVE;
          187  +#endif
   184    188         }
   185    189         break;
   186    190       }
   187    191       case SQLITE_MUTEX_FAST: {
   188    192         p = sqlite3MallocZero( sizeof(*p) );
   189    193         if( p ){
   190    194           pthread_mutex_init(&p->mutex, 0);
          195  +#if SQLITE_MUTEX_NREF || defined(SQLITE_ENABLE_API_ARMOR)
          196  +        p->id = SQLITE_MUTEX_FAST;
          197  +#endif
   191    198         }
   192    199         break;
   193    200       }
   194    201       default: {
   195    202   #ifdef SQLITE_ENABLE_API_ARMOR
   196    203         if( iType-2<0 || iType-2>=ArraySize(staticMutexes) ){
   197    204           (void)SQLITE_MISUSE_BKPT;
................................................................................
   199    206         }
   200    207   #endif
   201    208         p = &staticMutexes[iType-2];
   202    209         break;
   203    210       }
   204    211     }
   205    212   #if SQLITE_MUTEX_NREF || defined(SQLITE_ENABLE_API_ARMOR)
   206         -  if( p ) p->id = iType;
          213  +  assert( p==0 || p->id==iType );
   207    214   #endif
   208    215     return p;
   209    216   }
   210    217   
   211    218   
   212    219   /*
   213    220   ** This routine deallocates a previously

Changes to src/mutex_w32.c.

    36     36   */
    37     37   struct sqlite3_mutex {
    38     38     CRITICAL_SECTION mutex;    /* Mutex controlling the lock */
    39     39     int id;                    /* Mutex type */
    40     40   #ifdef SQLITE_DEBUG
    41     41     volatile int nRef;         /* Number of enterances */
    42     42     volatile DWORD owner;      /* Thread holding this mutex */
    43         -  volatile int trace;        /* True to trace changes */
           43  +  volatile LONG trace;       /* True to trace changes */
    44     44   #endif
    45     45   };
    46     46   
    47     47   /*
    48     48   ** These are the initializer values used when declaring a "static" mutex
    49     49   ** on Win32.  It should be noted that all mutexes require initialization
    50     50   ** on the Win32 platform.
    51     51   */
    52     52   #define SQLITE_W32_MUTEX_INITIALIZER { 0 }
    53     53   
    54     54   #ifdef SQLITE_DEBUG
    55         -#define SQLITE3_MUTEX_INITIALIZER { SQLITE_W32_MUTEX_INITIALIZER, 0, \
           55  +#define SQLITE3_MUTEX_INITIALIZER(id) { SQLITE_W32_MUTEX_INITIALIZER, id, \
    56     56                                       0L, (DWORD)0, 0 }
    57     57   #else
    58         -#define SQLITE3_MUTEX_INITIALIZER { SQLITE_W32_MUTEX_INITIALIZER, 0 }
           58  +#define SQLITE3_MUTEX_INITIALIZER(id) { SQLITE_W32_MUTEX_INITIALIZER, id }
    59     59   #endif
    60     60   
    61     61   #ifdef SQLITE_DEBUG
    62     62   /*
    63     63   ** The sqlite3_mutex_held() and sqlite3_mutex_notheld() routine are
    64     64   ** intended for use only inside assert() statements.
    65     65   */
................................................................................
    94     94   #endif
    95     95   }
    96     96   
    97     97   /*
    98     98   ** Initialize and deinitialize the mutex subsystem.
    99     99   */
   100    100   static sqlite3_mutex winMutex_staticMutexes[] = {
   101         -  SQLITE3_MUTEX_INITIALIZER,
   102         -  SQLITE3_MUTEX_INITIALIZER,
   103         -  SQLITE3_MUTEX_INITIALIZER,
   104         -  SQLITE3_MUTEX_INITIALIZER,
   105         -  SQLITE3_MUTEX_INITIALIZER,
   106         -  SQLITE3_MUTEX_INITIALIZER,
   107         -  SQLITE3_MUTEX_INITIALIZER,
   108         -  SQLITE3_MUTEX_INITIALIZER,
   109         -  SQLITE3_MUTEX_INITIALIZER,
   110         -  SQLITE3_MUTEX_INITIALIZER,
   111         -  SQLITE3_MUTEX_INITIALIZER,
   112         -  SQLITE3_MUTEX_INITIALIZER
          101  +  SQLITE3_MUTEX_INITIALIZER(2),
          102  +  SQLITE3_MUTEX_INITIALIZER(3),
          103  +  SQLITE3_MUTEX_INITIALIZER(4),
          104  +  SQLITE3_MUTEX_INITIALIZER(5),
          105  +  SQLITE3_MUTEX_INITIALIZER(6),
          106  +  SQLITE3_MUTEX_INITIALIZER(7),
          107  +  SQLITE3_MUTEX_INITIALIZER(8),
          108  +  SQLITE3_MUTEX_INITIALIZER(9),
          109  +  SQLITE3_MUTEX_INITIALIZER(10),
          110  +  SQLITE3_MUTEX_INITIALIZER(11),
          111  +  SQLITE3_MUTEX_INITIALIZER(12),
          112  +  SQLITE3_MUTEX_INITIALIZER(13)
   113    113   };
   114    114   
   115    115   static int winMutex_isInit = 0;
   116    116   static int winMutex_isNt = -1; /* <0 means "need to query" */
   117    117   
   118    118   /* As the winMutexInit() and winMutexEnd() functions are called as part
   119    119   ** of the sqlite3_initialize() and sqlite3_shutdown() processing, the
................................................................................
   235    235   #ifdef SQLITE_ENABLE_API_ARMOR
   236    236         if( iType-2<0 || iType-2>=ArraySize(winMutex_staticMutexes) ){
   237    237           (void)SQLITE_MISUSE_BKPT;
   238    238           return 0;
   239    239         }
   240    240   #endif
   241    241         p = &winMutex_staticMutexes[iType-2];
   242         -      p->id = iType;
   243    242   #ifdef SQLITE_DEBUG
   244    243   #ifdef SQLITE_WIN32_MUTEX_TRACE_STATIC
   245         -      p->trace = 1;
          244  +      InterlockedCompareExchange(&p->trace, 1, 0);
   246    245   #endif
   247    246   #endif
   248    247         break;
   249    248       }
   250    249     }
          250  +  assert( p==0 || p->id==iType );
   251    251     return p;
   252    252   }
   253    253   
   254    254   
   255    255   /*
   256    256   ** This routine deallocates a previously
   257    257   ** allocated mutex.  SQLite is careful to deallocate every

Changes to src/os_unix.c.

   464    464   #if defined(HAVE_FCHOWN)
   465    465     { "fchown",       (sqlite3_syscall_ptr)fchown,          0 },
   466    466   #else
   467    467     { "fchown",       (sqlite3_syscall_ptr)0,               0 },
   468    468   #endif
   469    469   #define osFchown    ((int(*)(int,uid_t,gid_t))aSyscall[20].pCurrent)
   470    470   
          471  +#if defined(HAVE_FCHOWN)
   471    472     { "geteuid",      (sqlite3_syscall_ptr)geteuid,         0 },
          473  +#else
          474  +  { "geteuid",      (sqlite3_syscall_ptr)0,               0 },
          475  +#endif
   472    476   #define osGeteuid   ((uid_t(*)(void))aSyscall[21].pCurrent)
   473    477   
   474    478   #if !defined(SQLITE_OMIT_WAL) || SQLITE_MAX_MMAP_SIZE>0
   475    479     { "mmap",         (sqlite3_syscall_ptr)mmap,            0 },
   476    480   #else
   477    481     { "mmap",         (sqlite3_syscall_ptr)0,               0 },
   478    482   #endif
................................................................................
  4172   4176     assert( n==1 || lockType!=F_RDLCK );
  4173   4177   
  4174   4178     /* Locks are within range */
  4175   4179     assert( n>=1 && n<=SQLITE_SHM_NLOCK );
  4176   4180   
  4177   4181     if( pShmNode->h>=0 ){
  4178   4182       /* Initialize the locking parameters */
  4179         -    memset(&f, 0, sizeof(f));
  4180   4183       f.l_type = lockType;
  4181   4184       f.l_whence = SEEK_SET;
  4182   4185       f.l_start = ofst;
  4183   4186       f.l_len = n;
  4184         -
  4185   4187       rc = osFcntl(pShmNode->h, F_SETLK, &f);
  4186   4188       rc = (rc!=(-1)) ? SQLITE_OK : SQLITE_BUSY;
  4187   4189     }
  4188   4190   
  4189   4191     /* Update the global lock state and do debug tracing */
  4190   4192   #ifdef SQLITE_DEBUG
  4191   4193     { u16 mask;

Changes to src/printf.c.

   202    202     double rounder;            /* Used for rounding floating point values */
   203    203     etByte flag_dp;            /* True if decimal point should be shown */
   204    204     etByte flag_rtz;           /* True if trailing zeros should be removed */
   205    205   #endif
   206    206     PrintfArguments *pArgList = 0; /* Arguments for SQLITE_PRINTF_SQLFUNC */
   207    207     char buf[etBUFSIZE];       /* Conversion buffer */
   208    208   
          209  +  /* pAccum never starts out with an empty buffer that was obtained from 
          210  +  ** malloc().  This precondition is required by the mprintf("%z...")
          211  +  ** optimization. */
          212  +  assert( pAccum->nChar>0 || (pAccum->printfFlags&SQLITE_PRINTF_MALLOCED)==0 );
          213  +
   209    214     bufpt = 0;
   210    215     if( (pAccum->printfFlags & SQLITE_PRINTF_SQLFUNC)!=0 ){
   211    216       pArgList = va_arg(ap, PrintfArguments*);
   212    217       bArgList = 1;
   213    218     }else{
   214    219       bArgList = 0;
   215    220     }
................................................................................
   620    625           buf[0] = '%';
   621    626           bufpt = buf;
   622    627           length = 1;
   623    628           break;
   624    629         case etCHARX:
   625    630           if( bArgList ){
   626    631             bufpt = getTextArg(pArgList);
   627         -          c = bufpt ? bufpt[0] : 0;
          632  +          length = 1;
          633  +          if( bufpt ){
          634  +            buf[0] = c = *(bufpt++);
          635  +            if( (c&0xc0)==0xc0 ){
          636  +              while( length<4 && (bufpt[0]&0xc0)==0x80 ){
          637  +                buf[length++] = *(bufpt++);
          638  +              }
          639  +            }
          640  +          }else{
          641  +            buf[0] = 0;
          642  +          }
   628    643           }else{
   629         -          c = va_arg(ap,int);
          644  +          unsigned int ch = va_arg(ap,unsigned int);
          645  +          if( ch<0x00080 ){
          646  +            buf[0] = ch & 0xff;
          647  +            length = 1;
          648  +          }else if( ch<0x00800 ){
          649  +            buf[0] = 0xc0 + (u8)((ch>>6)&0x1f);
          650  +            buf[1] = 0x80 + (u8)(ch & 0x3f);
          651  +            length = 2;
          652  +          }else if( ch<0x10000 ){
          653  +            buf[0] = 0xe0 + (u8)((ch>>12)&0x0f);
          654  +            buf[1] = 0x80 + (u8)((ch>>6) & 0x3f);
          655  +            buf[2] = 0x80 + (u8)(ch & 0x3f);
          656  +            length = 3;
          657  +          }else{
          658  +            buf[0] = 0xf0 + (u8)((ch>>18) & 0x07);
          659  +            buf[1] = 0x80 + (u8)((ch>>12) & 0x3f);
          660  +            buf[2] = 0x80 + (u8)((ch>>6) & 0x3f);
          661  +            buf[3] = 0x80 + (u8)(ch & 0x3f);
          662  +            length = 4;
          663  +          }
   630    664           }
   631    665           if( precision>1 ){
   632    666             width -= precision-1;
   633    667             if( width>1 && !flag_leftjustify ){
   634    668               sqlite3AppendChar(pAccum, width-1, ' ');
   635    669               width = 0;
   636    670             }
   637         -          sqlite3AppendChar(pAccum, precision-1, c);
          671  +          while( precision-- > 1 ){
          672  +            sqlite3StrAccumAppend(pAccum, buf, length);
          673  +          }
   638    674           }
   639         -        length = 1;
   640         -        buf[0] = c;
   641    675           bufpt = buf;
   642         -        break;
          676  +        flag_altform2 = 1;
          677  +        goto adjust_width_for_utf8;
   643    678         case etSTRING:
   644    679         case etDYNSTRING:
   645    680           if( bArgList ){
   646    681             bufpt = getTextArg(pArgList);
   647    682             xtype = etSTRING;
   648    683           }else{
   649    684             bufpt = va_arg(ap,char*);
   650    685           }
   651    686           if( bufpt==0 ){
   652    687             bufpt = "";
   653    688           }else if( xtype==etDYNSTRING ){
          689  +          if( pAccum->nChar==0 && pAccum->mxAlloc && width==0 && precision<0 ){
          690  +            /* Special optimization for sqlite3_mprintf("%z..."):
          691  +            ** Extend an existing memory allocation rather than creating
          692  +            ** a new one. */
          693  +            assert( (pAccum->printfFlags&SQLITE_PRINTF_MALLOCED)==0 );
          694  +            pAccum->zText = bufpt;
          695  +            pAccum->nAlloc = sqlite3DbMallocSize(pAccum->db, bufpt);
          696  +            pAccum->nChar = 0x7fffffff & (int)strlen(bufpt);
          697  +            pAccum->printfFlags |= SQLITE_PRINTF_MALLOCED;
          698  +            length = 0;
          699  +            break;
          700  +          }
   654    701             zExtra = bufpt;
   655    702           }
   656    703           if( precision>=0 ){
   657         -          for(length=0; length<precision && bufpt[length]; length++){}
          704  +          if( flag_altform2 ){
          705  +            /* Set length to the number of bytes needed in order to display
          706  +            ** precision characters */
          707  +            unsigned char *z = (unsigned char*)bufpt;
          708  +            while( precision-- > 0 && z[0] ){
          709  +              SQLITE_SKIP_UTF8(z);
          710  +            }
          711  +            length = (int)(z - (unsigned char*)bufpt);
          712  +          }else{
          713  +            for(length=0; length<precision && bufpt[length]; length++){}
          714  +          }
   658    715           }else{
   659    716             length = 0x7fffffff & (int)strlen(bufpt);
   660    717           }
          718  +      adjust_width_for_utf8:
          719  +        if( flag_altform2 && width>0 ){
          720  +          /* Adjust width to account for extra bytes in UTF-8 characters */
          721  +          int ii = length - 1;
          722  +          while( ii>=0 ) if( (bufpt[ii--] & 0xc0)==0x80 ) width++;
          723  +        }
   661    724           break;
   662         -      case etSQLESCAPE:           /* Escape ' characters */
   663         -      case etSQLESCAPE2:          /* Escape ' and enclose in '...' */
   664         -      case etSQLESCAPE3: {        /* Escape " characters */
          725  +      case etSQLESCAPE:           /* %q: Escape ' characters */
          726  +      case etSQLESCAPE2:          /* %Q: Escape ' and enclose in '...' */
          727  +      case etSQLESCAPE3: {        /* %w: Escape " characters */
   665    728           int i, j, k, n, isnull;
   666    729           int needQuote;
   667    730           char ch;
   668    731           char q = ((xtype==etSQLESCAPE3)?'"':'\'');   /* Quote character */
   669    732           char *escarg;
   670    733   
   671    734           if( bArgList ){
   672    735             escarg = getTextArg(pArgList);
   673    736           }else{
   674    737             escarg = va_arg(ap,char*);
   675    738           }
   676    739           isnull = escarg==0;
   677    740           if( isnull ) escarg = (xtype==etSQLESCAPE2 ? "NULL" : "(NULL)");
          741  +        /* For %q, %Q, and %w, the precision is the number of byte (or
          742  +        ** characters if the ! flags is present) to use from the input.
          743  +        ** Because of the extra quoting characters inserted, the number
          744  +        ** of output characters may be larger than the precision.
          745  +        */
   678    746           k = precision;
   679    747           for(i=n=0; k!=0 && (ch=escarg[i])!=0; i++, k--){
   680    748             if( ch==q )  n++;
          749  +          if( flag_altform2 && (ch&0xc0)==0xc0 ){
          750  +            while( (escarg[i+1]&0xc0)==0x80 ){ i++; }
          751  +          }
   681    752           }
   682    753           needQuote = !isnull && xtype==etSQLESCAPE2;
   683    754           n += i + 3;
   684    755           if( n>etBUFSIZE ){
   685    756             bufpt = zExtra = sqlite3Malloc( n );
   686    757             if( bufpt==0 ){
   687    758               setStrAccumError(pAccum, STRACCUM_NOMEM);
................................................................................
   696    767           for(i=0; i<k; i++){
   697    768             bufpt[j++] = ch = escarg[i];
   698    769             if( ch==q ) bufpt[j++] = ch;
   699    770           }
   700    771           if( needQuote ) bufpt[j++] = q;
   701    772           bufpt[j] = 0;
   702    773           length = j;
   703         -        /* The precision in %q and %Q means how many input characters to
   704         -        ** consume, not the length of the output...
   705         -        ** if( precision>=0 && precision<length ) length = precision; */
   706         -        break;
          774  +        goto adjust_width_for_utf8;
   707    775         }
   708    776         case etTOKEN: {
   709    777           Token *pToken;
   710    778           if( (pAccum->printfFlags & SQLITE_PRINTF_INTERNAL)==0 ) return;
   711    779           pToken = va_arg(ap, Token*);
   712    780           assert( bArgList==0 );
   713    781           if( pToken && pToken->n ){
................................................................................
   738    806           assert( xtype==etINVALID );
   739    807           return;
   740    808         }
   741    809       }/* End switch over the format type */
   742    810       /*
   743    811       ** The text of the conversion is pointed to by "bufpt" and is
   744    812       ** "length" characters long.  The field width is "width".  Do
   745         -    ** the output.
          813  +    ** the output.  Both length and width are in bytes, not characters,
          814  +    ** at this point.  If the "!" flag was present on string conversions
          815  +    ** indicating that width and precision should be expressed in characters,
          816  +    ** then the values have been translated prior to reaching this point.
   746    817       */
   747    818       width -= length;
   748    819       if( width>0 ){
   749    820         if( !flag_leftjustify ) sqlite3AppendChar(pAccum, width, ' ');
   750    821         sqlite3StrAccumAppend(pAccum, bufpt, length);
   751    822         if( flag_leftjustify ) sqlite3AppendChar(pAccum, width, ' ');
   752    823       }else{

Changes to src/shell.c.in.

  1065   1065   
  1066   1066   /* Allowed values for ShellState.openMode
  1067   1067   */
  1068   1068   #define SHELL_OPEN_UNSPEC     0      /* No open-mode specified */
  1069   1069   #define SHELL_OPEN_NORMAL     1      /* Normal database file */
  1070   1070   #define SHELL_OPEN_APPENDVFS  2      /* Use appendvfs */
  1071   1071   #define SHELL_OPEN_ZIPFILE    3      /* Use the zipfile virtual table */
         1072  +#define SHELL_OPEN_READONLY   4      /* Open a normal database read-only */
  1072   1073   
  1073   1074   /*
  1074   1075   ** These are the allowed shellFlgs values
  1075   1076   */
  1076   1077   #define SHFLG_Pagecache      0x00000001 /* The --pagecache option is used */
  1077   1078   #define SHFLG_Lookaside      0x00000002 /* Lookaside memory is used */
  1078   1079   #define SHFLG_Backslash      0x00000004 /* The --backslash option is used */
................................................................................
  3254   3255     "                         tcl      TCL list elements\n"
  3255   3256     ".nullvalue STRING      Use STRING in place of NULL values\n"
  3256   3257     ".once (-e|-x|FILE)     Output for the next SQL command only to FILE\n"
  3257   3258     "                         or invoke system text editor (-e) or spreadsheet (-x)\n"
  3258   3259     "                         on the output.\n"
  3259   3260     ".open ?OPTIONS? ?FILE? Close existing database and reopen FILE\n"
  3260   3261     "                         The --new option starts with an empty file\n"
         3262  +  "                         Other options: --readonly --append --zip\n"
  3261   3263     ".output ?FILE?         Send output to FILE or stdout\n"
  3262   3264     ".print STRING...       Print literal STRING\n"
  3263   3265     ".prompt MAIN CONTINUE  Replace the standard prompts\n"
  3264   3266     ".quit                  Exit this program\n"
  3265   3267     ".read FILENAME         Execute SQL in FILENAME\n"
  3266   3268     ".restore ?DB? FILE     Restore content of DB (default \"main\") from FILE\n"
  3267   3269     ".save FILE             Write in-memory database into FILE\n"
................................................................................
  3449   3451           sqlite3_open_v2(p->zDbFilename, &p->db, 
  3450   3452              SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, "apndvfs");
  3451   3453           break;
  3452   3454         }
  3453   3455         case SHELL_OPEN_ZIPFILE: {
  3454   3456           sqlite3_open(":memory:", &p->db);
  3455   3457           break;
         3458  +      }
         3459  +      case SHELL_OPEN_READONLY: {
         3460  +        sqlite3_open_v2(p->zDbFilename, &p->db, SQLITE_OPEN_READONLY, 0);
         3461  +        break;
  3456   3462         }
  3457   3463         case SHELL_OPEN_UNSPEC:
  3458   3464         case SHELL_OPEN_NORMAL: {
  3459   3465           sqlite3_open(p->zDbFilename, &p->db);
  3460   3466           break;
  3461   3467         }
  3462   3468       }
................................................................................
  6303   6309           newFlag = 1;
  6304   6310   #ifdef SQLITE_HAVE_ZIP
  6305   6311         }else if( optionMatch(z, "zip") ){
  6306   6312           p->openMode = SHELL_OPEN_ZIPFILE;
  6307   6313   #endif
  6308   6314         }else if( optionMatch(z, "append") ){
  6309   6315           p->openMode = SHELL_OPEN_APPENDVFS;
         6316  +      }else if( optionMatch(z, "readonly") ){
         6317  +        p->openMode = SHELL_OPEN_READONLY;
  6310   6318         }else if( z[0]=='-' ){
  6311   6319           utf8_printf(stderr, "unknown option: %s\n", z);
  6312   6320           rc = 1;
  6313   6321           goto meta_command_exit;
  6314   6322         }
  6315   6323       }
  6316   6324       /* If a filename is specified, try to open it first */
................................................................................
  7981   7989   #ifdef SQLITE_ENABLE_MULTIPLEX
  7982   7990     "   -multiplex           enable the multiplexor VFS\n"
  7983   7991   #endif
  7984   7992     "   -newline SEP         set output row separator. Default: '\\n'\n"
  7985   7993     "   -nullvalue TEXT      set text string for NULL values. Default ''\n"
  7986   7994     "   -pagecache SIZE N    use N slots of SZ bytes each for page cache memory\n"
  7987   7995     "   -quote               set output mode to 'quote'\n"
         7996  +  "   -readonly            open the database read-only\n"
  7988   7997     "   -separator SEP       set output column separator. Default: '|'\n"
  7989   7998     "   -stats               print memory stats before each finalize\n"
  7990   7999     "   -version             show SQLite version\n"
  7991   8000     "   -vfs NAME            use NAME as the default VFS\n"
  7992   8001   #ifdef SQLITE_ENABLE_VFSTRACE
  7993   8002     "   -vfstrace            enable tracing of all VFS calls\n"
  7994   8003   #endif
................................................................................
  8090   8099     if( strncmp(sqlite3_sourceid(),SQLITE_SOURCE_ID,60)!=0 ){
  8091   8100       utf8_printf(stderr, "SQLite header and source version mismatch\n%s\n%s\n",
  8092   8101               sqlite3_sourceid(), SQLITE_SOURCE_ID);
  8093   8102       exit(1);
  8094   8103     }
  8095   8104   #endif
  8096   8105     main_init(&data);
  8097         -  sqlite3_initialize();
         8106  +
         8107  +  /* On Windows, we must translate command-line arguments into UTF-8.
         8108  +  ** The SQLite memory allocator subsystem has to be enabled in order to
         8109  +  ** do this.  But we want to run an sqlite3_shutdown() afterwards so that
         8110  +  ** subsequent sqlite3_config() calls will work.  So copy all results into
         8111  +  ** memory that does not come from the SQLite memory allocator.
         8112  +  */
  8098   8113   #if !SQLITE_SHELL_IS_UTF8
  8099         -  argv = sqlite3_malloc64(sizeof(argv[0])*argc);
         8114  +  sqlite3_initialize();
         8115  +  argv = malloc(sizeof(argv[0])*argc);
  8100   8116     if( argv==0 ){
  8101   8117       raw_printf(stderr, "out of memory\n");
  8102   8118       exit(1);
  8103   8119     }
  8104   8120     for(i=0; i<argc; i++){
  8105         -    argv[i] = sqlite3_win32_unicode_to_utf8(wargv[i]);
         8121  +    char *z = sqlite3_win32_unicode_to_utf8(wargv[i]);
         8122  +    int n;
         8123  +    if( z==0 ){
         8124  +      raw_printf(stderr, "out of memory\n");
         8125  +      exit(1);
         8126  +    }
         8127  +    n = (int)strlen(z);
         8128  +    argv[i] = malloc( n+1 );
  8106   8129       if( argv[i]==0 ){
  8107   8130         raw_printf(stderr, "out of memory\n");
  8108   8131         exit(1);
  8109   8132       }
         8133  +    memcpy(argv[i], z, n+1);
         8134  +    sqlite3_free(z);
  8110   8135     }
         8136  +  sqlite3_shutdown();
  8111   8137   #endif
         8138  +
  8112   8139     assert( argc>=1 && argv && argv[0] );
  8113   8140     Argv0 = argv[0];
  8114   8141   
  8115   8142     /* Make sure we have a valid signal handler early, before anything
  8116   8143     ** else is done.
  8117   8144     */
  8118   8145   #ifdef SIGINT
................................................................................
  8230   8257         }
  8231   8258   #ifdef SQLITE_HAVE_ZIP
  8232   8259       }else if( strcmp(z,"-zip")==0 ){
  8233   8260         data.openMode = SHELL_OPEN_ZIPFILE;
  8234   8261   #endif
  8235   8262       }else if( strcmp(z,"-append")==0 ){
  8236   8263         data.openMode = SHELL_OPEN_APPENDVFS;
         8264  +    }else if( strcmp(z,"-readonly")==0 ){
         8265  +      data.openMode = SHELL_OPEN_READONLY;
  8237   8266       }
  8238   8267     }
  8239   8268     if( data.zDbFilename==0 ){
  8240   8269   #ifndef SQLITE_OMIT_MEMORYDB
  8241   8270       data.zDbFilename = ":memory:";
  8242   8271       warnInmemoryDb = argc==1;
  8243   8272   #else

Changes to src/sqlite.h.in.

  2492   2492   void sqlite3_free_table(char **result);
  2493   2493   
  2494   2494   /*
  2495   2495   ** CAPI3REF: Formatted String Printing Functions
  2496   2496   **
  2497   2497   ** These routines are work-alikes of the "printf()" family of functions
  2498   2498   ** from the standard C library.
  2499         -** These routines understand most of the common K&R formatting options,
  2500         -** plus some additional non-standard formats, detailed below.
  2501         -** Note that some of the more obscure formatting options from recent
  2502         -** C-library standards are omitted from this implementation.
         2499  +** These routines understand most of the common formatting options from
         2500  +** the standard library printf() 
         2501  +** plus some additional non-standard formats ([%q], [%Q], [%w], and [%z]).
         2502  +** See the [built-in printf()] documentation for details.
  2503   2503   **
  2504   2504   ** ^The sqlite3_mprintf() and sqlite3_vmprintf() routines write their
  2505         -** results into memory obtained from [sqlite3_malloc()].
         2505  +** results into memory obtained from [sqlite3_malloc64()].
  2506   2506   ** The strings returned by these two routines should be
  2507   2507   ** released by [sqlite3_free()].  ^Both routines return a
  2508         -** NULL pointer if [sqlite3_malloc()] is unable to allocate enough
         2508  +** NULL pointer if [sqlite3_malloc64()] is unable to allocate enough
  2509   2509   ** memory to hold the resulting string.
  2510   2510   **
  2511   2511   ** ^(The sqlite3_snprintf() routine is similar to "snprintf()" from
  2512   2512   ** the standard C library.  The result is written into the
  2513   2513   ** buffer supplied as the second parameter whose size is given by
  2514   2514   ** the first parameter. Note that the order of the
  2515   2515   ** first two parameters is reversed from snprintf().)^  This is an
................................................................................
  2525   2525   ** guarantees that the buffer is always zero-terminated.  ^The first
  2526   2526   ** parameter "n" is the total size of the buffer, including space for
  2527   2527   ** the zero terminator.  So the longest string that can be completely
  2528   2528   ** written will be n-1 characters.
  2529   2529   **
  2530   2530   ** ^The sqlite3_vsnprintf() routine is a varargs version of sqlite3_snprintf().
  2531   2531   **
  2532         -** These routines all implement some additional formatting
  2533         -** options that are useful for constructing SQL statements.
  2534         -** All of the usual printf() formatting options apply.  In addition, there
  2535         -** is are "%q", "%Q", "%w" and "%z" options.
  2536         -**
  2537         -** ^(The %q option works like %s in that it substitutes a nul-terminated
  2538         -** string from the argument list.  But %q also doubles every '\'' character.
  2539         -** %q is designed for use inside a string literal.)^  By doubling each '\''
  2540         -** character it escapes that character and allows it to be inserted into
  2541         -** the string.
  2542         -**
  2543         -** For example, assume the string variable zText contains text as follows:
  2544         -**
  2545         -** <blockquote><pre>
  2546         -**  char *zText = "It's a happy day!";
  2547         -** </pre></blockquote>
  2548         -**
  2549         -** One can use this text in an SQL statement as follows:
  2550         -**
  2551         -** <blockquote><pre>
  2552         -**  char *zSQL = sqlite3_mprintf("INSERT INTO table VALUES('%q')", zText);
  2553         -**  sqlite3_exec(db, zSQL, 0, 0, 0);
  2554         -**  sqlite3_free(zSQL);
  2555         -** </pre></blockquote>
  2556         -**
  2557         -** Because the %q format string is used, the '\'' character in zText
  2558         -** is escaped and the SQL generated is as follows:
  2559         -**
  2560         -** <blockquote><pre>
  2561         -**  INSERT INTO table1 VALUES('It''s a happy day!')
  2562         -** </pre></blockquote>
  2563         -**
  2564         -** This is correct.  Had we used %s instead of %q, the generated SQL
  2565         -** would have looked like this:
  2566         -**
  2567         -** <blockquote><pre>
  2568         -**  INSERT INTO table1 VALUES('It's a happy day!');
  2569         -** </pre></blockquote>
  2570         -**
  2571         -** This second example is an SQL syntax error.  As a general rule you should
  2572         -** always use %q instead of %s when inserting text into a string literal.
  2573         -**
  2574         -** ^(The %Q option works like %q except it also adds single quotes around
  2575         -** the outside of the total string.  Additionally, if the parameter in the
  2576         -** argument list is a NULL pointer, %Q substitutes the text "NULL" (without
  2577         -** single quotes).)^  So, for example, one could say:
  2578         -**
  2579         -** <blockquote><pre>
  2580         -**  char *zSQL = sqlite3_mprintf("INSERT INTO table VALUES(%Q)", zText);
  2581         -**  sqlite3_exec(db, zSQL, 0, 0, 0);
  2582         -**  sqlite3_free(zSQL);
  2583         -** </pre></blockquote>
  2584         -**
  2585         -** The code above will render a correct SQL statement in the zSQL
  2586         -** variable even if the zText variable is a NULL pointer.
  2587         -**
  2588         -** ^(The "%w" formatting option is like "%q" except that it expects to
  2589         -** be contained within double-quotes instead of single quotes, and it
  2590         -** escapes the double-quote character instead of the single-quote
  2591         -** character.)^  The "%w" formatting option is intended for safely inserting
  2592         -** table and column names into a constructed SQL statement.
  2593         -**
  2594         -** ^(The "%z" formatting option works like "%s" but with the
  2595         -** addition that after the string has been read and copied into
  2596         -** the result, [sqlite3_free()] is called on the input string.)^
         2532  +** See also:  [built-in printf()], [printf() SQL function]
  2597   2533   */
  2598   2534   char *sqlite3_mprintf(const char*,...);
  2599   2535   char *sqlite3_vmprintf(const char*, va_list);
  2600   2536   char *sqlite3_snprintf(int,char*,const char*, ...);
  2601   2537   char *sqlite3_vsnprintf(int,char*,const char*, va_list);
  2602   2538   
  2603   2539   /*

Changes to src/sqliteInt.h.

  4065   4065   extern const Token sqlite3IntTokens[];
  4066   4066   extern SQLITE_WSD struct Sqlite3Config sqlite3Config;
  4067   4067   extern FuncDefHash sqlite3BuiltinFunctions;
  4068   4068   #ifndef SQLITE_OMIT_WSD
  4069   4069   extern int sqlite3PendingByte;
  4070   4070   #endif
  4071   4071   #endif
         4072  +#ifdef VDBE_PROFILE
         4073  +extern sqlite3_uint64 sqlite3NProfileCnt;
         4074  +#endif
  4072   4075   void sqlite3RootPageMoved(sqlite3*, int, int, int);
  4073   4076   void sqlite3Reindex(Parse*, Token*, Token*);
  4074   4077   void sqlite3AlterFunctions(void);
  4075   4078   void sqlite3AlterRenameTable(Parse*, SrcList*, Token*);
  4076   4079   int sqlite3GetToken(const unsigned char *, int *);
  4077   4080   void sqlite3NestedParse(Parse*, const char*, ...);
  4078   4081   void sqlite3ExpirePreparedStatements(sqlite3*);

Changes to src/test_config.c.

   496    496   #ifdef SQLITE_ENABLE_MEMORY_MANAGEMENT
   497    497     Tcl_SetVar2(interp, "sqlite_options", "memorymanage", "1", TCL_GLOBAL_ONLY);
   498    498   #else
   499    499     Tcl_SetVar2(interp, "sqlite_options", "memorymanage", "0", TCL_GLOBAL_ONLY);
   500    500   #endif
   501    501   
   502    502   Tcl_SetVar2(interp, "sqlite_options", "mergesort", "1", TCL_GLOBAL_ONLY);
          503  +
          504  +#ifdef SQLITE_ENABLE_NULL_TRIM
          505  +  Tcl_SetVar2(interp, "sqlite_options", "null_trim", "1", TCL_GLOBAL_ONLY);
          506  +#else
          507  +  Tcl_SetVar2(interp, "sqlite_options", "null_trim", "0", TCL_GLOBAL_ONLY);
          508  +#endif
   503    509   
   504    510   #ifdef SQLITE_OMIT_OR_OPTIMIZATION
   505    511     Tcl_SetVar2(interp, "sqlite_options", "or_opt", "0", TCL_GLOBAL_ONLY);
   506    512   #else
   507    513     Tcl_SetVar2(interp, "sqlite_options", "or_opt", "1", TCL_GLOBAL_ONLY);
   508    514   #endif
   509    515   

Changes to src/vacuum.c.

    35     35   
    36     36     /* printf("SQL: [%s]\n", zSql); fflush(stdout); */
    37     37     rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
    38     38     if( rc!=SQLITE_OK ) return rc;
    39     39     while( SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){
    40     40       const char *zSubSql = (const char*)sqlite3_column_text(pStmt,0);
    41     41       assert( sqlite3_strnicmp(zSql,"SELECT",6)==0 );
    42         -    if( zSubSql ){
    43         -      assert( zSubSql[0]!='S' );
           42  +    assert( sqlite3_strnicmp(zSubSql,"SELECT",6)!=0 || CORRUPT_DB );
           43  +    if( zSubSql && zSubSql[0]!='S' ){
    44     44         rc = execSql(db, pzErrMsg, zSubSql);
    45     45         if( rc!=SQLITE_OK ) break;
    46     46       }
    47     47     }
    48     48     assert( rc!=SQLITE_ROW );
    49     49     if( rc==SQLITE_DONE ) rc = SQLITE_OK;
    50     50     if( rc ){

Changes to src/vdbe.c.

   644    644     for(pOp=&aOp[p->pc]; 1; pOp++){
   645    645       /* Errors are detected by individual opcodes, with an immediate
   646    646       ** jumps to abort_due_to_error. */
   647    647       assert( rc==SQLITE_OK );
   648    648   
   649    649       assert( pOp>=aOp && pOp<&aOp[p->nOp]);
   650    650   #ifdef VDBE_PROFILE
   651         -    start = sqlite3Hwtime();
          651  +    start = sqlite3NProfileCnt ? sqlite3NProfileCnt : sqlite3Hwtime();
   652    652   #endif
   653    653       nVmStep++;
   654    654   #ifdef SQLITE_ENABLE_STMT_SCANSTATUS
   655    655       if( p->anExec ) p->anExec[(int)(pOp-aOp)]++;
   656    656   #endif
   657    657   
   658    658       /* Only allow tracing if SQLITE_DEBUG is defined.
................................................................................
  7229   7229   ** readability.  From this point on down, the normal indentation rules are
  7230   7230   ** restored.
  7231   7231   *****************************************************************************/
  7232   7232       }
  7233   7233   
  7234   7234   #ifdef VDBE_PROFILE
  7235   7235       {
  7236         -      u64 endTime = sqlite3Hwtime();
         7236  +      u64 endTime = sqlite3NProfileCnt ? sqlite3NProfileCnt : sqlite3Hwtime();
  7237   7237         if( endTime>start ) pOrigOp->cycles += endTime - start;
  7238   7238         pOrigOp->cnt++;
  7239   7239       }
  7240   7240   #endif
  7241   7241   
  7242   7242       /* The following code adds nothing to the actual functionality
  7243   7243       ** of the program.  It is only here for testing and debugging.

Changes to src/wal.c.

   550    550   ** so.  It is safe to enlarge the wal-index if pWal->writeLock is true
   551    551   ** or pWal->exclusiveMode==WAL_HEAPMEMORY_MODE.
   552    552   **
   553    553   ** If this call is successful, *ppPage is set to point to the wal-index
   554    554   ** page and SQLITE_OK is returned. If an error (an OOM or VFS error) occurs,
   555    555   ** then an SQLite error code is returned and *ppPage is set to 0.
   556    556   */
   557         -static int walIndexPage(Wal *pWal, int iPage, volatile u32 **ppPage){
          557  +static SQLITE_NOINLINE int walIndexPageRealloc(
          558  +  Wal *pWal,               /* The WAL context */
          559  +  int iPage,               /* The page we seek */
          560  +  volatile u32 **ppPage    /* Write the page pointer here */
          561  +){
   558    562     int rc = SQLITE_OK;
   559    563   
   560    564     /* Enlarge the pWal->apWiData[] array if required */
   561    565     if( pWal->nWiData<=iPage ){
   562    566       int nByte = sizeof(u32*)*(iPage+1);
   563    567       volatile u32 **apNew;
   564    568       apNew = (volatile u32 **)sqlite3_realloc64((void *)pWal->apWiData, nByte);
................................................................................
   569    573       memset((void*)&apNew[pWal->nWiData], 0,
   570    574              sizeof(u32*)*(iPage+1-pWal->nWiData));
   571    575       pWal->apWiData = apNew;
   572    576       pWal->nWiData = iPage+1;
   573    577     }
   574    578   
   575    579     /* Request a pointer to the required page from the VFS */
   576         -  if( pWal->apWiData[iPage]==0 ){
   577         -    if( pWal->exclusiveMode==WAL_HEAPMEMORY_MODE ){
   578         -      pWal->apWiData[iPage] = (u32 volatile *)sqlite3MallocZero(WALINDEX_PGSZ);
   579         -      if( !pWal->apWiData[iPage] ) rc = SQLITE_NOMEM_BKPT;
   580         -    }else{
   581         -      rc = sqlite3OsShmMap(pWal->pDbFd, iPage, WALINDEX_PGSZ, 
   582         -          pWal->writeLock, (void volatile **)&pWal->apWiData[iPage]
   583         -      );
   584         -      assert( pWal->apWiData[iPage]!=0 || rc!=SQLITE_OK || pWal->writeLock==0 );
   585         -      testcase( pWal->apWiData[iPage]==0 && rc==SQLITE_OK );
   586         -      if( (rc&0xff)==SQLITE_READONLY ){
   587         -        pWal->readOnly |= WAL_SHM_RDONLY;
   588         -        if( rc==SQLITE_READONLY ){
   589         -          rc = SQLITE_OK;
   590         -        }
          580  +  assert( pWal->apWiData[iPage]==0 );
          581  +  if( pWal->exclusiveMode==WAL_HEAPMEMORY_MODE ){
          582  +    pWal->apWiData[iPage] = (u32 volatile *)sqlite3MallocZero(WALINDEX_PGSZ);
          583  +    if( !pWal->apWiData[iPage] ) rc = SQLITE_NOMEM_BKPT;
          584  +  }else{
          585  +    rc = sqlite3OsShmMap(pWal->pDbFd, iPage, WALINDEX_PGSZ, 
          586  +        pWal->writeLock, (void volatile **)&pWal->apWiData[iPage]
          587  +    );
          588  +    assert( pWal->apWiData[iPage]!=0 || rc!=SQLITE_OK || pWal->writeLock==0 );
          589  +    testcase( pWal->apWiData[iPage]==0 && rc==SQLITE_OK );
          590  +    if( (rc&0xff)==SQLITE_READONLY ){
          591  +      pWal->readOnly |= WAL_SHM_RDONLY;
          592  +      if( rc==SQLITE_READONLY ){
          593  +        rc = SQLITE_OK;
   591    594         }
   592    595       }
   593    596     }
   594    597   
   595    598     *ppPage = pWal->apWiData[iPage];
   596    599     assert( iPage==0 || *ppPage || rc!=SQLITE_OK );
   597    600     return rc;
          601  +}
          602  +static int walIndexPage(
          603  +  Wal *pWal,               /* The WAL context */
          604  +  int iPage,               /* The page we seek */
          605  +  volatile u32 **ppPage    /* Write the page pointer here */
          606  +){
          607  +  if( pWal->nWiData<=iPage || (*ppPage = pWal->apWiData[iPage])==0 ){
          608  +    return walIndexPageRealloc(pWal, iPage, ppPage);
          609  +  }
          610  +  return SQLITE_OK;
   598    611   }
   599    612   
   600    613   /*
   601    614   ** Return a pointer to the WalCkptInfo structure in the wal-index.
   602    615   */
   603    616   static volatile WalCkptInfo *walCkptInfo(Wal *pWal){
   604    617     assert( pWal->nWiData>0 && pWal->apWiData[0] );
................................................................................
  2864   2877     **     This condition filters out normal hash-table collisions.
  2865   2878     **
  2866   2879     **   (iFrame<=iLast): 
  2867   2880     **     This condition filters out entries that were added to the hash
  2868   2881     **     table after the current read-transaction had started.
  2869   2882     */
  2870   2883     iMinHash = walFramePage(pWal->minFrame);
  2871         -  for(iHash=walFramePage(iLast); iHash>=iMinHash && iRead==0; iHash--){
         2884  +  for(iHash=walFramePage(iLast); iHash>=iMinHash; iHash--){
  2872   2885       volatile ht_slot *aHash;      /* Pointer to hash table */
  2873   2886       volatile u32 *aPgno;          /* Pointer to array of page numbers */
  2874   2887       u32 iZero;                    /* Frame number corresponding to aPgno[0] */
  2875   2888       int iKey;                     /* Hash slot index */
  2876   2889       int nCollide;                 /* Number of hash collisions remaining */
  2877   2890       int rc;                       /* Error code */
  2878   2891   
................................................................................
  2887   2900           assert( iFrame>iRead || CORRUPT_DB );
  2888   2901           iRead = iFrame;
  2889   2902         }
  2890   2903         if( (nCollide--)==0 ){
  2891   2904           return SQLITE_CORRUPT_BKPT;
  2892   2905         }
  2893   2906       }
         2907  +    if( iRead ) break;
  2894   2908     }
  2895   2909   
  2896   2910   #ifdef SQLITE_ENABLE_EXPENSIVE_ASSERT
  2897   2911     /* If expensive assert() statements are available, do a linear search
  2898   2912     ** of the wal-index file content. Make sure the results agree with the
  2899   2913     ** result obtained using the hash indexes above.  */
  2900   2914     {

Changes to src/wherecode.c.

  1391   1391         testcase( pStart->wtFlags & TERM_VIRTUAL );
  1392   1392         pX = pStart->pExpr;
  1393   1393         assert( pX!=0 );
  1394   1394         testcase( pStart->leftCursor!=iCur ); /* transitive constraints */
  1395   1395         if( sqlite3ExprIsVector(pX->pRight) ){
  1396   1396           r1 = rTemp = sqlite3GetTempReg(pParse);
  1397   1397           codeExprOrVector(pParse, pX->pRight, r1, 1);
  1398         -        op = aMoveOp[(pX->op - TK_GT) | 0x0001];
         1398  +        testcase( pX->op==TK_GT );
         1399  +        testcase( pX->op==TK_GE );
         1400  +        testcase( pX->op==TK_LT );
         1401  +        testcase( pX->op==TK_LE );
         1402  +        op = aMoveOp[((pX->op - TK_GT - 1) & 0x3) | 0x1];
         1403  +        assert( pX->op!=TK_GT || op==OP_SeekGE );
         1404  +        assert( pX->op!=TK_GE || op==OP_SeekGE );
         1405  +        assert( pX->op!=TK_LT || op==OP_SeekLE );
         1406  +        assert( pX->op!=TK_LE || op==OP_SeekLE );
  1399   1407         }else{
  1400   1408           r1 = sqlite3ExprCodeTemp(pParse, pX->pRight, &rTemp);
  1401   1409           disableTerm(pLevel, pStart);
  1402   1410           op = aMoveOp[(pX->op - TK_GT)];
  1403   1411         }
  1404   1412         sqlite3VdbeAddOp3(v, op, iCur, addrBrk, r1);
  1405   1413         VdbeComment((v, "pk"));

Changes to src/whereexpr.c.

  1287   1287   
  1288   1288         pNew = sqlite3PExpr(pParse, pExpr->op, pLeft, pRight);
  1289   1289         transferJoinMarkings(pNew, pExpr);
  1290   1290         idxNew = whereClauseInsert(pWC, pNew, TERM_DYNAMIC);
  1291   1291         exprAnalyze(pSrc, pWC, idxNew);
  1292   1292       }
  1293   1293       pTerm = &pWC->a[idxTerm];
  1294         -    pTerm->wtFlags = TERM_CODED|TERM_VIRTUAL;  /* Disable the original */
         1294  +    pTerm->wtFlags |= TERM_CODED|TERM_VIRTUAL;  /* Disable the original */
  1295   1295       pTerm->eOperator = 0;
  1296   1296     }
  1297   1297   
  1298   1298     /* If there is a vector IN term - e.g. "(a, b) IN (SELECT ...)" - create
  1299   1299     ** a virtual term for each vector component. The expression object
  1300   1300     ** used by each such virtual term is pExpr (the full vector IN(...) 
  1301   1301     ** expression). The WhereTerm.iField variable identifies the index within

Changes to test/fts4onepass.test.

   138    138       do_execsql_test  3.$tn.$tn2.b { SELECT rowid, content FROM ft2 } $content
   139    139       do_execsql_test  3.$tn.$tn2.c { 
   140    140         INSERT INTO ft2(ft2) VALUES('integrity-check');
   141    141       }
   142    142     }
   143    143     eval $tcl2
   144    144   }
          145  +
          146  +do_execsql_test 4.0 {
          147  +  CREATE VIRTUAL TABLE zt USING fts4(a, b);
          148  +  INSERT INTO zt(rowid, a, b) VALUES(1, 'unus duo', NULL);
          149  +  INSERT INTO zt(rowid, a, b) VALUES(2, NULL, NULL);
          150  +
          151  +  BEGIN;
          152  +    UPDATE zt SET b='septum' WHERE rowid = 1;
          153  +    UPDATE zt SET b='octo' WHERE rowid = 1;
          154  +  COMMIT;
          155  +
          156  +  SELECT count(*) FROM zt_segdir;
          157  +} {3}
          158  +
   145    159   
   146    160   finish_test

Changes to test/func6.test.

    17     17   #
    18     18   set testdir [file dirname $argv0]
    19     19   source $testdir/tester.tcl
    20     20   ifcapable !offset_sql_func {
    21     21     finish_test
    22     22     return
    23     23   }
           24  +
           25  +set bNullTrim 0
           26  +ifcapable null_trim {
           27  +  set bNullTrim 1
           28  +}
    24     29   
    25     30   do_execsql_test func6-100 {
    26     31     PRAGMA page_size=4096;
    27     32     PRAGMA auto_vacuum=NONE;
    28     33     CREATE TABLE t1(a,b,c,d);
    29     34     WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x<100)
    30     35      INSERT INTO t1(a,b,c,d) SELECT printf('abc%03x',x), x, 1000-x, NULL FROM c;
................................................................................
    47     52   # Each argument is either an integer between 0 and 65535, a text value, or
    48     53   # an empty string representing an SQL NULL. This command builds an SQLite
    49     54   # record containing the values passed as arguments and returns it encoded
    50     55   # as a hex string.
    51     56   proc hexrecord {args} {
    52     57     set hdr ""
    53     58     set body ""
           59  +
           60  +  if {$::bNullTrim} {
           61  +    while {[llength $args] && [lindex $args end]=={}} {
           62  +      set args [lrange $args 0 end-1]
           63  +    }
           64  +  }
           65  +
    54     66     foreach x $args {
    55     67       if {$x==""} {
    56     68         append hdr 00
    57     69       } elseif {[string is integer $x]==0} {
    58     70         set n [string length $x]
    59     71         append hdr [format %02x [expr $n*2 + 13]]
    60     72         append body [binary encode hex $x]
................................................................................
    99    111     set rec [hexrecord {*}[lrange $args 1 end]]
   100    112     offset_contains_record $offset $::F $rec
   101    113   }
   102    114   set F [loadhex test.db]
   103    115   db func offrec offrec
   104    116   
   105    117   # Test the sanity of the tests.
          118  +if {$bNullTrim} {
          119  +  set offset 8180
          120  +} else {
          121  +  set offset 8179
          122  +}
   106    123   do_execsql_test func6-105 {
   107    124     SELECT sqlite_offset(d) FROM t1 ORDER BY rowid LIMIT 1;
   108         -} {8179}
          125  +} $offset
   109    126   do_test func6-106 {
   110    127     set r [hexrecord abc001 1 999 {}]
   111         -  offset_contains_record 8179 $F $r
          128  +  offset_contains_record $offset $F $r
   112    129   } 0
   113    130   
   114    131   set z100 [string trim [string repeat "0 " 100]]
   115    132   
   116    133   # Test offsets within table b-tree t1.
   117    134   do_execsql_test func6-110 {
   118    135     SELECT offrec(sqlite_offset(d), a, b, c, d) FROM t1 ORDER BY rowid

Changes to test/printf2.test.

   144    144   do_execsql_test printf2-4.9 {
   145    145     SELECT printf('|%,d|%,d|',123456789,-123456789);
   146    146   } {|123,456,789|-123,456,789|}
   147    147   do_execsql_test printf2-4.10 {
   148    148     SELECT printf('|%,d|%,d|',1234567890,-1234567890);
   149    149   } {|1,234,567,890|-1,234,567,890|}
   150    150   
          151  +# 2018-02-19.  Unicode characters with %c
          152  +do_execsql_test printf2-5.100 {
          153  +  SELECT printf('(%8c)',char(11106));
          154  +} {{(       ⭢)}}
          155  +do_execsql_test printf2-5.101 {
          156  +  SELECT printf('(%-8c)',char(11106));
          157  +} {{(⭢       )}}
          158  +do_execsql_test printf2-5.102 {
          159  +  SELECT printf('(%5.3c)',char(1492));
          160  +} {{(  ההה)}}
          161  +do_execsql_test printf2-5.103 {
          162  +  SELECT printf('(%-5.3c)',char(1492));
          163  +} {{(ההה  )}}
          164  +do_execsql_test printf2-5.104 {
          165  +  SELECT printf('(%3.3c)',char(1492));
          166  +} {{(ההה)}}
          167  +do_execsql_test printf2-5.105 {
          168  +  SELECT printf('(%-3.3c)',char(1492));
          169  +} {{(ההה)}}
          170  +do_execsql_test printf2-5.104 {
          171  +  SELECT printf('(%2c)',char(1513));
          172  +} {{( ש)}}
          173  +do_execsql_test printf2-5.106 {
          174  +  SELECT printf('(%-2c)',char(1513));
          175  +} {{(ש )}}
          176  +
          177  +# 2018-02-19.  Unicode characters with the "!" flag in %s and friends.
          178  +do_execsql_test printf2-6.100 {
          179  +  SELECT printf('(%!.3s)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          180  +} {(הנה)}
          181  +do_execsql_test printf2-6.101 {
          182  +  SELECT printf('(%.6s)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          183  +} {(הנה)}
          184  +do_execsql_test printf2-6.102 {
          185  +  SELECT printf('(%!5.3s)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          186  +} {{(  הנה)}}
          187  +do_execsql_test printf2-6.103 {
          188  +  SELECT printf('(%8.6s)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          189  +} {{(  הנה)}}
          190  +do_execsql_test printf2-6.104 {
          191  +  SELECT printf('(%!-5.3s)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          192  +} {{(הנה  )}}
          193  +do_execsql_test printf2-6.105 {
          194  +  SELECT printf('(%-8.6s)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          195  +} {{(הנה  )}}
          196  +do_execsql_test printf2-6.106 {
          197  +  SELECT printf('(%!.3Q)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          198  +} {('הנה')}
          199  +do_execsql_test printf2-6.107 {
          200  +  SELECT printf('(%.6Q)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          201  +} {('הנה')}
          202  +do_execsql_test printf2-6.108 {
          203  +  SELECT printf('(%!7.3Q)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          204  +} {{(  'הנה')}}
          205  +do_execsql_test printf2-6.109 {
          206  +  SELECT printf('(%10.6Q)','הנה מה־טוב ומה־נעים שבת אחים גם־יחד');
          207  +} {{(  'הנה')}}
   151    208   
   152    209   
   153    210   finish_test

Changes to test/rowvalue.test.

   433    433   } {1 1 1 1 2 1}
   434    434   do_execsql_test 18.6 {
   435    435     SELECT * FROM b3 JOIN b4 ON b4.a = b3.a
   436    436     WHERE (b3.a, b3.b) IN ( SELECT a, b FROM b5 ); 
   437    437   } {1 1 1 1 2 1}
   438    438   
   439    439    
   440         -finish_test
          440  +# 2018-02-13 Ticket https://www.sqlite.org/src/tktview/f484b65f3d6230593c3
          441  +# Incorrect result from a row-value comparison in the WHERE clause.
          442  +#
          443  +do_execsql_test 19.1 {
          444  +  DROP TABLE IF EXISTS t1;
          445  +  CREATE TABLE t1(a INTEGER PRIMARY KEY,b);
          446  +  INSERT INTO t1(a,b) VALUES(1,11),(2,22),(3,33),(4,44);
          447  +  SELECT * FROM t1 WHERE (a,b)>(0,0) ORDER BY a;
          448  +} {1 11 2 22 3 33 4 44}
          449  +do_execsql_test 19.2 {
          450  +  SELECT * FROM t1 WHERE (a,b)>=(0,0) ORDER BY a;
          451  +} {1 11 2 22 3 33 4 44}
          452  +do_execsql_test 19.3 {
          453  +  SELECT * FROM t1 WHERE (a,b)<(5,0) ORDER BY a DESC;
          454  +} {4 44 3 33 2 22 1 11}
          455  +do_execsql_test 19.4 {
          456  +  SELECT * FROM t1 WHERE (a,b)<=(5,0) ORDER BY a DESC;
          457  +} {4 44 3 33 2 22 1 11}
          458  +do_execsql_test 19.5 {
          459  +  SELECT * FROM t1 WHERE (a,b)>(3,0) ORDER BY a;
          460  +} {3 33 4 44}
          461  +do_execsql_test 19.6 {
          462  +  SELECT * FROM t1 WHERE (a,b)>=(3,0) ORDER BY a;
          463  +} {3 33 4 44}
          464  +do_execsql_test 19.7 {
          465  +  SELECT * FROM t1 WHERE (a,b)<(3,0) ORDER BY a DESC;
          466  +} {2 22 1 11}
          467  +do_execsql_test 19.8 {
          468  +  SELECT * FROM t1 WHERE (a,b)<=(3,0) ORDER BY a DESC;
          469  +} {2 22 1 11}
          470  +do_execsql_test 19.9 {
          471  +  SELECT * FROM t1 WHERE (a,b)>(3,32) ORDER BY a;
          472  +} {3 33 4 44}
          473  +do_execsql_test 19.10 {
          474  +  SELECT * FROM t1 WHERE (a,b)>(3,33) ORDER BY a;
          475  +} {4 44}
          476  +do_execsql_test 19.11 {
          477  +  SELECT * FROM t1 WHERE (a,b)>=(3,33) ORDER BY a;
          478  +} {3 33 4 44}
          479  +do_execsql_test 19.12 {
          480  +  SELECT * FROM t1 WHERE (a,b)>=(3,34) ORDER BY a;
          481  +} {4 44}
          482  +do_execsql_test 19.13 {
          483  +  SELECT * FROM t1 WHERE (a,b)<(3,34) ORDER BY a DESC;
          484  +} {3 33 2 22 1 11}
          485  +do_execsql_test 19.14 {
          486  +  SELECT * FROM t1 WHERE (a,b)<(3,33) ORDER BY a DESC;
          487  +} {2 22 1 11}
          488  +do_execsql_test 19.15 {
          489  +  SELECT * FROM t1 WHERE (a,b)<=(3,33) ORDER BY a DESC;
          490  +} {3 33 2 22 1 11}
          491  +do_execsql_test 19.16 {
          492  +  SELECT * FROM t1 WHERE (a,b)<=(3,32) ORDER BY a DESC;
          493  +} {2 22 1 11}
          494  +do_execsql_test 19.21 {
          495  +  SELECT * FROM t1 WHERE (0,0)<(a,b) ORDER BY a;
          496  +} {1 11 2 22 3 33 4 44}
          497  +do_execsql_test 19.22 {
          498  +  SELECT * FROM t1 WHERE (0,0)<=(a,b) ORDER BY a;
          499  +} {1 11 2 22 3 33 4 44}
          500  +do_execsql_test 19.23 {
          501  +  SELECT * FROM t1 WHERE (5,0)>(a,b) ORDER BY a DESC;
          502  +} {4 44 3 33 2 22 1 11}
          503  +do_execsql_test 19.24 {
          504  +  SELECT * FROM t1 WHERE (5,0)>=(a,b) ORDER BY a DESC;
          505  +} {4 44 3 33 2 22 1 11}
          506  +do_execsql_test 19.25 {
          507  +  SELECT * FROM t1 WHERE (3,0)<(a,b) ORDER BY a;
          508  +} {3 33 4 44}
          509  +do_execsql_test 19.26 {
          510  +  SELECT * FROM t1 WHERE (3,0)<=(a,b) ORDER BY a;
          511  +} {3 33 4 44}
          512  +do_execsql_test 19.27 {
          513  +  SELECT * FROM t1 WHERE (3,0)>(a,b) ORDER BY a DESC;
          514  +} {2 22 1 11}
          515  +do_execsql_test 19.28 {
          516  +  SELECT * FROM t1 WHERE (3,0)>=(a,b) ORDER BY a DESC;
          517  +} {2 22 1 11}
          518  +do_execsql_test 19.29 {
          519  +  SELECT * FROM t1 WHERE (3,32)<(a,b) ORDER BY a;
          520  +} {3 33 4 44}
          521  +do_execsql_test 19.30 {
          522  +  SELECT * FROM t1 WHERE (3,33)<(a,b) ORDER BY a;
          523  +} {4 44}
          524  +do_execsql_test 19.31 {
          525  +  SELECT * FROM t1 WHERE (3,33)<=(a,b) ORDER BY a;
          526  +} {3 33 4 44}
          527  +do_execsql_test 19.32 {
          528  +  SELECT * FROM t1 WHERE (3,34)<=(a,b) ORDER BY a;
          529  +} {4 44}
          530  +do_execsql_test 19.33 {
          531  +  SELECT * FROM t1 WHERE (3,34)>(a,b) ORDER BY a DESC;
          532  +} {3 33 2 22 1 11}
          533  +do_execsql_test 19.34 {
          534  +  SELECT * FROM t1 WHERE (3,33)>(a,b) ORDER BY a DESC;
          535  +} {2 22 1 11}
          536  +do_execsql_test 19.35 {
          537  +  SELECT * FROM t1 WHERE (3,33)>=(a,b) ORDER BY a DESC;
          538  +} {3 33 2 22 1 11}
          539  +do_execsql_test 19.36 {
          540  +  SELECT * FROM t1 WHERE (3,32)>=(a,b) ORDER BY a DESC;
          541  +} {2 22 1 11}
          542  +
          543  +# 2018-02-18: Memory leak nexted row-value.  Detected by OSSFuzz.
          544  +#
          545  +do_catchsql_test 20.1 {
          546  +  SELECT 1 WHERE (2,(2,0)) IS (2,(2,0));
          547  +} {0 1}
   441    548   
   442    549   finish_test

Changes to test/spellfix.test.

   275    275       SELECT word, distance FROM t3 WHERE rowid = 10;
   276    276     } {keener {}
   277    277       {SELECT word, rank, NULL, langid, id FROM "main"."t3_vocab" WHERE rowid=?}
   278    278     }
   279    279     do_tracesql_test 6.2.3 {
   280    280       SELECT word, distance FROM t3 WHERE rowid = 10 AND word MATCH 'kiiner';
   281    281     } {keener 300
   282         -    {SELECT id, word, rank, k1  FROM "main"."t3_vocab" WHERE langid=0 AND k2>=?1 AND k2<?2}
          282  +    {SELECT id, word, rank, coalesce(k1,word)  FROM "main"."t3_vocab" WHERE langid=0 AND k2>=?1 AND k2<?2}
   283    283     }
   284    284   }
   285    285   
   286    286   #------------------------------------------------------------------------- 
   287    287   # Test that the spellfix1 table supports conflict handling (OR REPLACE 
   288    288   # and so on).
   289    289   #

Added test/spellfix4.test.

            1  +# 2018-02-14
            2  +#
            3  +# The author disclaims copyright to this source code.  In place of
            4  +# a legal notice, here is a blessing:
            5  +#
            6  +#    May you do good and not evil.
            7  +#    May you find forgiveness for yourself and forgive others.
            8  +#    May you share freely, never taking more than you give.
            9  +#
           10  +#***********************************************************************
           11  +#
           12  +# Test cases for the editdist3() function in the spellfix extension.
           13  +#
           14  +
           15  +set testdir [file dirname $argv0]
           16  +source $testdir/tester.tcl
           17  +set testprefix spellfix4
           18  +
           19  +ifcapable !vtab { finish_test ; return }
           20  +
           21  +load_static_extension db spellfix
           22  +
           23  +do_execsql_test 100 {
           24  +  CREATE TABLE cost1(iLang, cFrom, cTo, iCost);
           25  +  INSERT INTO cost1 VALUES
           26  +    (0, '', '?',  97),
           27  +    (0, '?', '',  98),
           28  +    (0, '?', '?', 99),
           29  +    (0, 'm', 'n', 50),
           30  +    (0, 'n', 'm', 50)
           31  +  ;
           32  +  SELECT editdist3('cost1');
           33  +  SELECT editdist3('anchor','amchor');
           34  +} {{} 50}
           35  +do_execsql_test 110 {
           36  +  SELECT editdist3('anchor','anchoxr');
           37  +} {97}
           38  +do_execsql_test 111 {
           39  +  SELECT editdist3('anchor','xanchor');
           40  +} {97}
           41  +do_execsql_test 112 {
           42  +  SELECT editdist3('anchor','anchorx');
           43  +} {97}
           44  +do_execsql_test 120 {
           45  +  SELECT editdist3('anchor','anchr');
           46  +} {98}
           47  +do_execsql_test 121 {
           48  +  SELECT editdist3('anchor','ancho');
           49  +} {98}
           50  +do_execsql_test 122 {
           51  +  SELECT editdist3('anchor','nchor');
           52  +} {98}
           53  +do_execsql_test 130 {
           54  +  SELECT editdist3('anchor','anchur');
           55  +} {99}
           56  +do_execsql_test 131 {
           57  +  SELECT editdist3('anchor','onchor');
           58  +} {99}
           59  +do_execsql_test 132 {
           60  +  SELECT editdist3('anchor','anchot');
           61  +} {99}
           62  +do_execsql_test 140 {
           63  +  SELECT editdist3('anchor','omchor');
           64  +} {149}
           65  +
           66  +do_execsql_test 200 {
           67  +  INSERT INTO cost1 VALUES
           68  +    (0, 'a', 'ä', 5),
           69  +    (0, 'ss', 'ß', 8)
           70  +  ;
           71  +  SELECT editdist3('cost1');
           72  +  SELECT editdist3('strasse','straße');
           73  +  SELECT editdist3('straße','strasse');
           74  +} {{} 8 196}
           75  +do_execsql_test 210 {
           76  +  SELECT editdist3('baume','bäume');
           77  +} {5}
           78  +do_execsql_test 220 {
           79  +  SELECT editdist3('baum','bäume');
           80  +} {102}
           81  +do_execsql_test 230 {
           82  +  INSERT INTO cost1 VALUES
           83  +    (0, 'ä', 'a', 5),
           84  +    (0, 'ß', 'ss', 8)
           85  +  ;
           86  +  SELECT editdist3('cost1');
           87  +  SELECT editdist3('strasse','straße');
           88  +  SELECT editdist3('straße','strasse');
           89  +} {{} 8 8}
           90  +
           91  +do_execsql_test 300 {
           92  +  DELETE FROM cost1;
           93  +  INSERT INTO cost1 VALUES
           94  +    (0, '', '?',  97),
           95  +    (0, '?', '',  98),
           96  +    (0, '?', '?', 99),
           97  +    (0, 'a', 'e', 50),
           98  +    (0, 'a', 'i', 70),
           99  +    (0, 'a', 'o', 75),
          100  +    (0, 'a', 'u', 81),
          101  +    (0, 'e', 'a', 50),
          102  +    (0, 'e', 'i', 52),
          103  +    (0, 'e', 'o', 72),
          104  +    (0, 'e', 'u', 82),
          105  +    (0, 'i', 'a', 70),
          106  +    (0, 'i', 'e', 52),
          107  +    (0, 'i', 'o', 75),
          108  +    (0, 'i', 'u', 83),
          109  +    (0, 'o', 'a', 75),
          110  +    (0, 'o', 'e', 72),
          111  +    (0, 'o', 'i', 75),
          112  +    (0, 'o', 'u', 40),
          113  +    (0, 'u', 'a', 81),
          114  +    (0, 'u', 'e', 82),
          115  +    (0, 'u', 'i', 83),
          116  +    (0, 'u', 'o', 40),
          117  +    (0, 'm', 'n', 45),
          118  +    (0, 'n', 'm', 45)
          119  +  ;
          120  +  CREATE TABLE words(x TEXT);
          121  +  INSERT INTO words VALUES
          122  +   ('abraham'),
          123  +   ('action'),
          124  +   ('africa'),
          125  +   ('aladdin'),
          126  +   ('alert'),
          127  +   ('alien'),
          128  +   ('amazon'),
          129  +   ('analog'),
          130  +   ('animal'),
          131  +   ('apollo'),
          132  +   ('archive'),
          133  +   ('arnold'),
          134  +   ('aspirin'),
          135  +   ('august'),
          136  +   ('average'),
          137  +   ('bahama'),
          138  +   ('bambino'),
          139  +   ('barcode'),
          140  +   ('bazooka'),
          141  +   ('belgium'),
          142  +   ('between'),
          143  +   ('biology'),
          144  +   ('blonde'),
          145  +   ('border'),
          146  +   ('brave'),
          147  +   ('british'),
          148  +   ('bucket'),
          149  +   ('button'),
          150  +   ('caesar'),
          151  +   ('camilla'),
          152  +   ('cannon'),
          153  +   ('caramel'),
          154  +   ('carpet'),
          155  +   ('catalog'),
          156  +   ('century'),
          157  +   ('chaos'),
          158  +   ('chef'),
          159  +   ('china'),
          160  +   ('circus'),
          161  +   ('classic'),
          162  +   ('clinic'),
          163  +   ('coconut'),
          164  +   ('combine'),
          165  +   ('complex'),
          166  +   ('congo'),
          167  +   ('convert'),
          168  +   ('cosmos'),
          169  +   ('crack'),
          170  +   ('crown'),
          171  +   ('cyclone'),
          172  +   ('deal'),
          173  +   ('delete'),
          174  +   ('denver'),
          175  +   ('detail'),
          176  +   ('diana'),
          177  +   ('direct'),
          178  +   ('dolby'),
          179  +   ('double'),
          180  +   ('dublin'),
          181  +   ('echo'),
          182  +   ('edition'),
          183  +   ('electra'),
          184  +   ('emotion'),
          185  +   ('enjoy'),
          186  +   ('escape'),
          187  +   ('everest'),
          188  +   ('exile'),
          189  +   ('express'),
          190  +   ('family'),
          191  +   ('ferrari'),
          192  +   ('filter'),
          193  +   ('fish'),
          194  +   ('florida'),
          195  +   ('ford'),
          196  +   ('forum'),
          197  +   ('frank'),
          198  +   ('frozen'),
          199  +   ('gallery'),
          200  +   ('garlic'),
          201  +   ('geneva'),
          202  +   ('gibson'),
          203  +   ('gloria'),
          204  +   ('gordon'),
          205  +   ('gravity'),
          206  +   ('ground'),
          207  +   ('habitat'),
          208  +   ('harlem'),
          209  +   ('hazard'),
          210  +   ('herbert'),
          211  +   ('hobby'),
          212  +   ('house'),
          213  +   ('icon'),
          214  +   ('immune'),
          215  +   ('india'),
          216  +   ('inside'),
          217  +   ('isotope'),
          218  +   ('jamaica'),
          219  +   ('jazz'),
          220  +   ('joker'),
          221  +   ('juliet'),
          222  +   ('jupiter'),
          223  +   ('kevin'),
          224  +   ('korea'),
          225  +   ('latin'),
          226  +   ('legal'),
          227  +   ('lexicon'),
          228  +   ('limbo'),
          229  +   ('lithium'),
          230  +   ('logo'),
          231  +   ('lucas'),
          232  +   ('madrid'),
          233  +   ('major'),
          234  +   ('manual'),
          235  +   ('mars'),
          236  +   ('maximum'),
          237  +   ('medical'),
          238  +   ('mental'),
          239  +   ('meter'),
          240  +   ('miguel'),
          241  +   ('mimosa'),
          242  +   ('miranda'),
          243  +   ('modern'),
          244  +   ('money'),
          245  +   ('morgan'),
          246  +   ('motor'),
          247  +   ('mystic'),
          248  +   ('nebula'),
          249  +   ('network'),
          250  +   ('nice'),
          251  +   ('nitro'),
          252  +   ('norway'),
          253  +   ('nurse'),
          254  +   ('octavia'),
          255  +   ('olympic'),
          256  +   ('opus'),
          257  +   ('orient'),
          258  +   ('othello'),
          259  +   ('pacific'),
          260  +   ('panama'),
          261  +   ('paper'),
          262  +   ('parking'),
          263  +   ('pasta'),
          264  +   ('paul'),
          265  +   ('people'),
          266  +   ('permit'),
          267  +   ('phrase'),
          268  +   ('pilgrim'),
          269  +   ('planet'),
          270  +   ('pocket'),
          271  +   ('police'),
          272  +   ('popular'),
          273  +   ('prefer'),
          274  +   ('presto'),
          275  +   ('private'),
          276  +   ('project'),
          277  +   ('proxy'),
          278  +   ('python'),
          279  +   ('quota'),
          280  +   ('rainbow'),
          281  +   ('raymond'),
          282  +   ('region'),
          283  +   ('report'),
          284  +   ('reward'),
          285  +   ('risk'),
          286  +   ('robot'),
          287  +   ('rose'),
          288  +   ('russian'),
          289  +   ('sailor'),
          290  +   ('salt'),
          291  +   ('saturn'),
          292  +   ('scorpio'),
          293  +   ('second'),
          294  +   ('seminar'),
          295  +   ('shadow'),
          296  +   ('shave'),
          297  +   ('shock'),
          298  +   ('silence'),
          299  +   ('sinatra'),
          300  +   ('sleep'),
          301  +   ('social'),
          302  +   ('sonata'),
          303  +   ('spain'),
          304  +   ('sphere'),
          305  +   ('spray'),
          306  +   ('state'),
          307  +   ('stone'),
          308  +   ('strong'),
          309  +   ('sugar'),
          310  +   ('supreme'),
          311  +   ('swing'),
          312  +   ('talent'),
          313  +   ('telecom'),
          314  +   ('thermos'),
          315  +   ('tina'),
          316  +   ('tommy'),
          317  +   ('torso'),
          318  +   ('trade'),
          319  +   ('trick'),
          320  +   ('tropic'),
          321  +   ('turtle'),
          322  +   ('uniform'),
          323  +   ('user'),
          324  +   ('vega'),
          325  +   ('vertigo'),
          326  +   ('village'),
          327  +   ('visible'),
          328  +   ('vocal'),
          329  +   ('voyage'),
          330  +   ('weekend'),
          331  +   ('winter'),
          332  +   ('year'),
          333  +   ('zipper')
          334  +  ;
          335  +  SELECT editdist3('cost1');
          336  +} {{}}
          337  +do_execsql_test 310 {
          338  +  SELECT editdist3(a.x,b.x), a.x, b.x
          339  +    FROM words a, words b
          340  +   WHERE a.x<b.x
          341  +   ORDER BY 1, 2
          342  +   LIMIT 20
          343  +} {139 bucket pocket 144 meter motor 149 manual mental 169 crack trick 173 sinatra sonata 174 edition emotion 174 major motor 174 risk rose 174 state stone 194 deal detail 196 alert talent 196 analog catalog 196 deal legal 196 ford forum 196 risk trick 196 stone strong 197 china tina 197 congo logo 197 diana tina 197 florida gloria}
          344  +do_execsql_test 320 {
          345  +  SELECT md5sum(ed||'/'||sx||'/'||sy||',') FROM (
          346  +      SELECT editdist3(a.x,b.x) AS ed, a.x AS sx, b.x AS sy
          347  +        FROM words a, words b
          348  +       WHERE a.x<b.x
          349  +       ORDER BY 1, 2
          350  +  )
          351  +} {69d0a31872203a775e19325ea98cd053}
          352  +
          353  +finish_test

Added test/with4.test.

            1  +# 2018-02-15
            2  +#
            3  +# The author disclaims copyright to this source code.  In place of
            4  +# a legal notice, here is a blessing:
            5  +#
            6  +#    May you do good and not evil.
            7  +#    May you find forgiveness for yourself and forgive others.
            8  +#    May you share freely, never taking more than you give.
            9  +#
           10  +#***********************************************************************
           11  +# This file implements regression tests for SQLite library.  The
           12  +# focus of this file is testing the WITH clause in TRIGGERs and VIEWs.
           13  +#
           14  +
           15  +set testdir [file dirname $argv0]
           16  +source $testdir/tester.tcl
           17  +set ::testprefix with4
           18  +
           19  +ifcapable {!cte} {
           20  +  finish_test
           21  +  return
           22  +}
           23  +
           24  +do_execsql_test 100 {
           25  +  ATTACH ':memory:' AS aux;
           26  +  CREATE TABLE main.t1(a,b);
           27  +  CREATE TABLE aux.t2(x,y);
           28  +  INSERT INTO t1 VALUES(1,2);
           29  +  INSERT INTO t2 VALUES(3,4);
           30  +} {}
           31  +do_catchsql_test 110 {
           32  +  CREATE VIEW v1 AS SELECT * FROM t1, aux.t2;
           33  +} {1 {view v1 cannot reference objects in database aux}}
           34  +do_catchsql_test 120 {
           35  +  CREATE VIEW v2 AS WITH v(m,n) AS (SELECT x,y FROM aux.t2) SELECT * FROM t1, v;
           36  +} {1 {view v2 cannot reference objects in database aux}}
           37  +do_catchsql_test 130 {
           38  +  CREATE VIEW v2 AS WITH v(m,n) AS (SELECT 5,?2) SELECT * FROM t1, v;
           39  +} {1 {parameters are not allowed in views}}
           40  +
           41  +do_catchsql_test 200 {
           42  +  CREATE TRIGGER r1 AFTER INSERT ON t1 BEGIN
           43  +     WITH v(m,n) AS (SELECT x,y FROM aux.t2) SELECT * FROM t1, v;
           44  +  END;
           45  +} {1 {trigger r1 cannot reference objects in database aux}}
           46  +do_catchsql_test 210 {
           47  +  CREATE TRIGGER r1 AFTER INSERT ON t1 BEGIN
           48  +     WITH v(m,n) AS (SELECT 5,?2) SELECT * FROM t1, v;
           49  +  END;
           50  +} {1 {trigger cannot use variables}}
           51  +
           52  +finish_test

Changes to tool/speed-check.sh.

    35     35   LEAN_OPTS="$LEAN_OPTS -DSQLITE_OMIT_DEPRECATED"
    36     36   LEAN_OPTS="$LEAN_OPTS -DSQLITE_OMIT_PROGRESS_CALLBACK"
    37     37   LEAN_OPTS="$LEAN_OPTS -DSQLITE_OMIT_SHARED_CACHE"
    38     38   LEAN_OPTS="$LEAN_OPTS -DSQLITE_USE_ALLOCA"
    39     39   BASELINE="trunk"
    40     40   doExplain=0
    41     41   doCachegrind=1
           42  +doVdbeProfile=0
           43  +doWal=1
    42     44   while test "$1" != ""; do
    43     45     case $1 in
    44     46       --reprepare)
    45     47           SPEEDTEST_OPTS="$SPEEDTEST_OPTS $1"
    46     48           ;;
    47     49       --autovacuum)
    48     50           SPEEDTEST_OPTS="$SPEEDTEST_OPTS $1"
................................................................................
    57     59           SPEEDTEST_OPTS="$SPEEDTEST_OPTS $1"
    58     60           ;;
    59     61       --nomemstat)
    60     62           SPEEDTEST_OPTS="$SPEEDTEST_OPTS $1"
    61     63           ;;
    62     64       --temp)
    63     65           SPEEDTEST_OPTS="$SPEEDTEST_OPTS --temp 6"
           66  +        ;;
           67  +    --legacy)
           68  +	doWal=0
    64     69           ;;
    65     70       --wal)
    66         -        SPEEDTEST_OPTS="$SPEEDTEST_OPTS --journal wal"
           71  +        doWal=1
    67     72           ;;
    68     73       --size)
    69     74           shift; SIZE=$1
    70     75           ;;
    71     76       --cachesize)
    72     77           shift; SPEEDTEST_OPTS="$SPEEDTEST_OPTS --cachesize $1"
    73     78           ;;
................................................................................
    74     79       --explain)
    75     80           doExplain=1
    76     81           ;;
    77     82       --vdbeprofile)
    78     83           rm -f vdbe_profile.out
    79     84           CC_OPTS="$CC_OPTS -DVDBE_PROFILE"
    80     85           doCachegrind=0
           86  +        doVdbeProfile=1
    81     87           ;;
    82     88       --lean)
    83     89           CC_OPTS="$CC_OPTS $LEAN_OPTS"
    84     90           ;;
    85     91       --clang)
    86     92           CC=clang
    87     93           ;;
................................................................................
   128    134           ;;
   129    135       *)
   130    136   	BASELINE=$1
   131    137           ;;
   132    138     esac
   133    139     shift
   134    140   done
          141  +if test $doWal -eq 1; then
          142  +  SPEEDTEST_OPTS="$SPEEDTEST_OPTS --journal wal"
          143  +fi
   135    144   SPEEDTEST_OPTS="$SPEEDTEST_OPTS --size $SIZE"
   136    145   echo "NAME           = $NAME" | tee summary-$NAME.txt
   137    146   echo "SPEEDTEST_OPTS = $SPEEDTEST_OPTS" | tee -a summary-$NAME.txt
   138    147   echo "CC_OPTS        = $CC_OPTS" | tee -a summary-$NAME.txt
   139    148   rm -f cachegrind.out.* speedtest1 speedtest1.db sqlite3.o
          149  +if test $doVdbeProfile -eq 1; then
          150  +  rm -f vdbe_profile.out
          151  +fi
   140    152   $CC -g -Os -Wall -I. $CC_OPTS -c sqlite3.c
   141    153   size sqlite3.o | tee -a summary-$NAME.txt
   142    154   if test $doExplain -eq 1; then
   143    155     $CC -g -Os -Wall -I. $CC_OPTS \
   144    156        -DSQLITE_ENABLE_EXPLAIN_COMMENTS \
   145    157       ./shell.c ./sqlite3.c -o sqlite3 -ldl -lpthread
   146    158   fi
................................................................................
   159    171     cg_anno.tcl cachegrind.out.* >cout-$NAME.txt
   160    172     echo '*****************************************************' >>cout-$NAME.txt
   161    173     sed 's/^[0-9=-]\{9\}/==00000==/' summary-$NAME.txt >>cout-$NAME.txt
   162    174   fi
   163    175   if test $doExplain -eq 1; then
   164    176     ./speedtest1 --explain $SPEEDTEST_OPTS | ./sqlite3 >explain-$NAME.txt
   165    177   fi
   166         -if test "$NAME" != "$BASELINE"; then
          178  +if test $doVdbeProfile -eq 1; then
          179  +  tclsh ../sqlite/tool/vdbe_profile.tcl >vdbeprofile-$NAME.txt
          180  +  open vdbeprofile-$NAME.txt
          181  +fi
          182  +if test "$NAME" != "$BASELINE" -a $doVdbeProfile -ne 1; then
   167    183     fossil test-diff --tk -c 20 cout-$BASELINE.txt cout-$NAME.txt
   168    184   fi