/ Check-in [53b14a38]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merge all recent trunk enhancements into the apple-osx branch.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | apple-osx
Files: files | file ages | folders
SHA3-256: 53b14a388838439087f92282bdaa3f706e9e197a7ed93f8c73e8844cffca8126
User & Date: drh 2017-06-23 21:05:45
Context
2017-06-27
16:48
Merge latest trunk changes with this branch. check-in: 2b095406 user: dan tags: apple-osx
2017-06-23
21:05
Merge all recent trunk enhancements into the apple-osx branch. check-in: 53b14a38 user: drh tags: apple-osx
15:47
When generating individual loops for each ORed term of an OR scan, move any constant WHERE expressions outside of the loop, as is done for top-level loops. check-in: 712267c9 user: dan tags: trunk
2017-06-08
14:41
Merge the auto_vacuum bug fix and all other changes from the 3.19.3 release. check-in: 93f32dd2 user: drh tags: apple-osx
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to VERSION.

     1         -3.19.3
            1  +3.20.0

Changes to autoconf/Makefile.am.

     8      8   bin_PROGRAMS = sqlite3
     9      9   sqlite3_SOURCES = shell.c sqlite3.h
    10     10   EXTRA_sqlite3_SOURCES = sqlite3.c
    11     11   sqlite3_LDADD = @EXTRA_SHELL_OBJ@ @READLINE_LIBS@
    12     12   sqlite3_DEPENDENCIES = @EXTRA_SHELL_OBJ@
    13     13   sqlite3_CFLAGS = $(AM_CFLAGS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS
    14     14   
    15         -include_HEADERS = sqlite3.h sqlite3ext.h msvc.h
           15  +include_HEADERS = sqlite3.h sqlite3ext.h
    16     16   
    17     17   EXTRA_DIST = sqlite3.1 tea Makefile.msc sqlite3.rc README.txt Replace.cs
    18     18   pkgconfigdir = ${libdir}/pkgconfig
    19     19   pkgconfig_DATA = sqlite3.pc
    20     20   
    21     21   man_MANS = sqlite3.1

Changes to configure.

     1      1   #! /bin/sh
     2      2   # Guess values for system-dependent variables and create Makefiles.
     3         -# Generated by GNU Autoconf 2.69 for sqlite 3.19.3.
            3  +# Generated by GNU Autoconf 2.69 for sqlite 3.20.0.
     4      4   #
     5      5   #
     6      6   # Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.
     7      7   #
     8      8   #
     9      9   # This configure script is free software; the Free Software Foundation
    10     10   # gives unlimited permission to copy, distribute and modify it.
................................................................................
   722    722   subdirs=
   723    723   MFLAGS=
   724    724   MAKEFLAGS=
   725    725   
   726    726   # Identity of this package.
   727    727   PACKAGE_NAME='sqlite'
   728    728   PACKAGE_TARNAME='sqlite'
   729         -PACKAGE_VERSION='3.19.3'
   730         -PACKAGE_STRING='sqlite 3.19.3'
          729  +PACKAGE_VERSION='3.20.0'
          730  +PACKAGE_STRING='sqlite 3.20.0'
   731    731   PACKAGE_BUGREPORT=''
   732    732   PACKAGE_URL=''
   733    733   
   734    734   # Factoring default headers for most tests.
   735    735   ac_includes_default="\
   736    736   #include <stdio.h>
   737    737   #ifdef HAVE_SYS_TYPES_H
................................................................................
  1459   1459   #
  1460   1460   # Report the --help message.
  1461   1461   #
  1462   1462   if test "$ac_init_help" = "long"; then
  1463   1463     # Omit some internal or obsolete options to make the list less imposing.
  1464   1464     # This message is too long to be a string in the A/UX 3.1 sh.
  1465   1465     cat <<_ACEOF
  1466         -\`configure' configures sqlite 3.19.3 to adapt to many kinds of systems.
         1466  +\`configure' configures sqlite 3.20.0 to adapt to many kinds of systems.
  1467   1467   
  1468   1468   Usage: $0 [OPTION]... [VAR=VALUE]...
  1469   1469   
  1470   1470   To assign environment variables (e.g., CC, CFLAGS...), specify them as
  1471   1471   VAR=VALUE.  See below for descriptions of some of the useful variables.
  1472   1472   
  1473   1473   Defaults for the options are specified in brackets.
................................................................................
  1524   1524     --build=BUILD     configure for building on BUILD [guessed]
  1525   1525     --host=HOST       cross-compile to build programs to run on HOST [BUILD]
  1526   1526   _ACEOF
  1527   1527   fi
  1528   1528   
  1529   1529   if test -n "$ac_init_help"; then
  1530   1530     case $ac_init_help in
  1531         -     short | recursive ) echo "Configuration of sqlite 3.19.3:";;
         1531  +     short | recursive ) echo "Configuration of sqlite 3.20.0:";;
  1532   1532      esac
  1533   1533     cat <<\_ACEOF
  1534   1534   
  1535   1535   Optional Features:
  1536   1536     --disable-option-checking  ignore unrecognized --enable/--with options
  1537   1537     --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)
  1538   1538     --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]
................................................................................
  1648   1648       cd "$ac_pwd" || { ac_status=$?; break; }
  1649   1649     done
  1650   1650   fi
  1651   1651   
  1652   1652   test -n "$ac_init_help" && exit $ac_status
  1653   1653   if $ac_init_version; then
  1654   1654     cat <<\_ACEOF
  1655         -sqlite configure 3.19.3
         1655  +sqlite configure 3.20.0
  1656   1656   generated by GNU Autoconf 2.69
  1657   1657   
  1658   1658   Copyright (C) 2012 Free Software Foundation, Inc.
  1659   1659   This configure script is free software; the Free Software Foundation
  1660   1660   gives unlimited permission to copy, distribute and modify it.
  1661   1661   _ACEOF
  1662   1662     exit
................................................................................
  2067   2067     eval $as_lineno_stack; ${as_lineno_stack:+:} unset as_lineno
  2068   2068   
  2069   2069   } # ac_fn_c_check_header_mongrel
  2070   2070   cat >config.log <<_ACEOF
  2071   2071   This file contains any messages produced by compilers while
  2072   2072   running configure, to aid debugging if configure makes a mistake.
  2073   2073   
  2074         -It was created by sqlite $as_me 3.19.3, which was
         2074  +It was created by sqlite $as_me 3.20.0, which was
  2075   2075   generated by GNU Autoconf 2.69.  Invocation command line was
  2076   2076   
  2077   2077     $ $0 $@
  2078   2078   
  2079   2079   _ACEOF
  2080   2080   exec 5>>config.log
  2081   2081   {
................................................................................
 12147  12147   test $as_write_fail = 0 && chmod +x $CONFIG_STATUS || ac_write_fail=1
 12148  12148   
 12149  12149   cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
 12150  12150   # Save the log message, to keep $0 and so on meaningful, and to
 12151  12151   # report actual input values of CONFIG_FILES etc. instead of their
 12152  12152   # values after options handling.
 12153  12153   ac_log="
 12154         -This file was extended by sqlite $as_me 3.19.3, which was
        12154  +This file was extended by sqlite $as_me 3.20.0, which was
 12155  12155   generated by GNU Autoconf 2.69.  Invocation command line was
 12156  12156   
 12157  12157     CONFIG_FILES    = $CONFIG_FILES
 12158  12158     CONFIG_HEADERS  = $CONFIG_HEADERS
 12159  12159     CONFIG_LINKS    = $CONFIG_LINKS
 12160  12160     CONFIG_COMMANDS = $CONFIG_COMMANDS
 12161  12161     $ $0 $@
................................................................................
 12213  12213   
 12214  12214   Report bugs to the package provider."
 12215  12215   
 12216  12216   _ACEOF
 12217  12217   cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1
 12218  12218   ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`"
 12219  12219   ac_cs_version="\\
 12220         -sqlite config.status 3.19.3
        12220  +sqlite config.status 3.20.0
 12221  12221   configured by $0, generated by GNU Autoconf 2.69,
 12222  12222     with options \\"\$ac_cs_config\\"
 12223  12223   
 12224  12224   Copyright (C) 2012 Free Software Foundation, Inc.
 12225  12225   This config.status script is free software; the Free Software Foundation
 12226  12226   gives unlimited permission to copy, distribute and modify it."
 12227  12227   

Changes to doc/lemon.html.

    18     18   Lemon also implements features that can be used
    19     19   to eliminate resource leaks, making is suitable for use
    20     20   in long-running programs such as graphical user interfaces
    21     21   or embedded controllers.</p>
    22     22   
    23     23   <p>This document is an introduction to the Lemon
    24     24   parser generator.</p>
           25  +
           26  +<h2>Security Note</h2>
           27  +
           28  +<p>The language parser code created by Lemon is very robust and
           29  +is well-suited for use in internet-facing applications that need to
           30  +safely process maliciously crafted inputs.
           31  +
           32  +<p>The "lemon.exe" command-line tool itself works great when given a valid
           33  +input grammar file and almost always gives helpful
           34  +error messages for malformed inputs.  However,  it is possible for
           35  +a malicious user to craft a grammar file that will cause 
           36  +lemon.exe to crash.
           37  +We do not see this as a problem, as lemon.exe is not intended to be used
           38  +with hostile inputs.
           39  +To summarize:</p>
           40  +
           41  +<ul>
           42  +<li>Parser code generated by lemon &rarr; Robust and secure
           43  +<li>The "lemon.exe" command line tool itself &rarr; Not so much
           44  +</ul>
    25     45   
    26     46   <h2>Theory of Operation</h2>
    27     47   
    28     48   <p>The main goal of Lemon is to translate a context free grammar (CFG)
    29     49   for a particular language into C code that implements a parser for
    30     50   that language.
    31     51   The program has two inputs:

Changes to ext/fts5/fts5_hash.c.

   167    167   
   168    168     apNew = (Fts5HashEntry**)sqlite3_malloc(nNew*sizeof(Fts5HashEntry*));
   169    169     if( !apNew ) return SQLITE_NOMEM;
   170    170     memset(apNew, 0, nNew*sizeof(Fts5HashEntry*));
   171    171   
   172    172     for(i=0; i<pHash->nSlot; i++){
   173    173       while( apOld[i] ){
   174         -      int iHash;
          174  +      unsigned int iHash;
   175    175         Fts5HashEntry *p = apOld[i];
   176    176         apOld[i] = p->pHashNext;
   177         -      iHash = fts5HashKey(nNew, (u8*)fts5EntryKey(p), strlen(fts5EntryKey(p)));
          177  +      iHash = fts5HashKey(nNew, (u8*)fts5EntryKey(p),
          178  +                          (int)strlen(fts5EntryKey(p)));
   178    179         p->pHashNext = apNew[iHash];
   179    180         apNew[iHash] = p;
   180    181       }
   181    182     }
   182    183   
   183    184     sqlite3_free(apOld);
   184    185     pHash->nSlot = nNew;
................................................................................
   473    474   int sqlite3Fts5HashQuery(
   474    475     Fts5Hash *pHash,                /* Hash table to query */
   475    476     const char *pTerm, int nTerm,   /* Query term */
   476    477     const u8 **ppDoclist,           /* OUT: Pointer to doclist for pTerm */
   477    478     int *pnDoclist                  /* OUT: Size of doclist in bytes */
   478    479   ){
   479    480     unsigned int iHash = fts5HashKey(pHash->nSlot, (const u8*)pTerm, nTerm);
   480         -  char *zKey;
          481  +  char *zKey = 0;
   481    482     Fts5HashEntry *p;
   482    483   
   483    484     for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){
   484    485       zKey = fts5EntryKey(p);
   485    486       if( memcmp(zKey, pTerm, nTerm)==0 && zKey[nTerm]==0 ) break;
   486    487     }
   487    488   

Changes to ext/fts5/fts5_test_tok.c.

   178    178     sqlite3_vtab **ppVtab,          /* OUT: New sqlite3_vtab object */
   179    179     char **pzErr                    /* OUT: sqlite3_malloc'd error message */
   180    180   ){
   181    181     fts5_api *pApi = (fts5_api*)pCtx;
   182    182     Fts5tokTable *pTab = 0;
   183    183     int rc;
   184    184     char **azDequote = 0;
   185         -  int nDequote;
          185  +  int nDequote = 0;
   186    186   
   187    187     rc = sqlite3_declare_vtab(db, 
   188    188          "CREATE TABLE x(input HIDDEN, token, start, end, position)"
   189    189     );
   190    190   
   191    191     if( rc==SQLITE_OK ){
   192    192       nDequote = argc-3;

Changes to ext/fts5/test/fts5rank.test.

    86     86     execsql { SELECT rowid FROM tt('a') ORDER BY rank; } db2
    87     87   } {1 3 2}
    88     88   
    89     89   do_test 2.7 {
    90     90     execsql { SELECT rowid FROM tt('a') ORDER BY rank; } db
    91     91   } {1 3 2}
    92     92   
           93  +db2 close
    93     94   
    94     95   #--------------------------------------------------------------------------
    95     96   # At one point there was a problem with queries such as:
    96     97   #
    97     98   #   ... MATCH 'x OR y' ORDER BY rank;
    98     99   #
    99    100   # if there were zero occurrences of token 'y' in the dataset. The

Changes to ext/misc/amatch.c.

   997    997     amatchEncodeInt(pWord->iSeq, pWord->zCost+4);
   998    998     pWord->zCost[8] = 0;
   999    999   }
  1000   1000   
  1001   1001   /* Circumvent compiler warnings about the use of strcpy() by supplying
  1002   1002   ** our own implementation.
  1003   1003   */
  1004         -#if defined(__OpenBSD__)
  1005   1004   static void amatchStrcpy(char *dest, const char *src){
  1006   1005     while( (*(dest++) = *(src++))!=0 ){}
  1007   1006   }
  1008   1007   static void amatchStrcat(char *dest, const char *src){
  1009   1008     while( *dest ) dest++;
  1010   1009     amatchStrcpy(dest, src);
  1011   1010   }
  1012         -#else
  1013         -# define amatchStrcpy strcpy
  1014         -# define amatchStrcat strcat
  1015         -#endif
  1016         -
  1017   1011   
  1018   1012   /*
  1019   1013   ** Add a new amatch_word object to the queue.
  1020   1014   **
  1021   1015   ** If a prior amatch_word object with the same zWord, and nMatch
  1022   1016   ** already exists, update its rCost (if the new rCost is less) but
  1023   1017   ** otherwise leave it unchanged.  Do not add a duplicate.

Changes to ext/misc/series.c.

    29     29   ** Integers 20 through 29.
    30     30   **
    31     31   ** HOW IT WORKS
    32     32   **
    33     33   ** The generate_series "function" is really a virtual table with the
    34     34   ** following schema:
    35     35   **
    36         -**     CREATE FUNCTION generate_series(
           36  +**     CREATE TABLE generate_series(
    37     37   **       value,
    38     38   **       start HIDDEN,
    39     39   **       stop HIDDEN,
    40     40   **       step HIDDEN
    41     41   **     );
    42     42   **
    43     43   ** Function arguments in queries against this virtual table are translated

Added ext/misc/stmts.c.

            1  +/*
            2  +** 2017-05-31
            3  +**
            4  +** The author disclaims copyright to this source code.  In place of
            5  +** a legal notice, here is a blessing:
            6  +**
            7  +**    May you do good and not evil.
            8  +**    May you find forgiveness for yourself and forgive others.
            9  +**    May you share freely, never taking more than you give.
           10  +**
           11  +*************************************************************************
           12  +**
           13  +** This file demonstrates an eponymous virtual table that returns information
           14  +** about all prepared statements for the database connection.
           15  +**
           16  +** Usage example:
           17  +**
           18  +**     .load ./stmts
           19  +**     .mode line
           20  +**     .header on
           21  +**     SELECT * FROM stmts;
           22  +*/
           23  +#include "sqlite3ext.h"
           24  +SQLITE_EXTENSION_INIT1
           25  +#include <assert.h>
           26  +#include <string.h>
           27  +
           28  +#ifndef SQLITE_OMIT_VIRTUALTABLE
           29  +
           30  +/*
           31  +** The following macros are used to cast pointers to integers.
           32  +** The way you do this varies from one compiler
           33  +** to the next, so we have developed the following set of #if statements
           34  +** to generate appropriate macros for a wide range of compilers.
           35  +*/
           36  +#if defined(__PTRDIFF_TYPE__)  /* This case should work for GCC */
           37  +# define SQLITE_PTR_TO_INT64(X)  ((sqlite3_int64)(__PTRDIFF_TYPE__)(X))
           38  +#elif !defined(__GNUC__)       /* Works for compilers other than LLVM */
           39  +# define SQLITE_PTR_TO_INT64(X)  ((sqlite3_int64)(((char*)X)-(char*)0))
           40  +#elif defined(HAVE_STDINT_H)   /* Use this case if we have ANSI headers */
           41  +# define SQLITE_PTR_TO_INT64(X)  ((sqlite3_int64)(intptr_t)(X))
           42  +#else                          /* Generates a warning - but it always works */
           43  +# define SQLITE_PTR_TO_INT64(X)  ((sqlite3_int64)(X))
           44  +#endif
           45  +
           46  +
           47  +/* stmts_vtab is a subclass of sqlite3_vtab which will
           48  +** serve as the underlying representation of a stmts virtual table
           49  +*/
           50  +typedef struct stmts_vtab stmts_vtab;
           51  +struct stmts_vtab {
           52  +  sqlite3_vtab base;  /* Base class - must be first */
           53  +  sqlite3 *db;        /* Database connection for this stmts vtab */
           54  +};
           55  +
           56  +/* stmts_cursor is a subclass of sqlite3_vtab_cursor which will
           57  +** serve as the underlying representation of a cursor that scans
           58  +** over rows of the result
           59  +*/
           60  +typedef struct stmts_cursor stmts_cursor;
           61  +struct stmts_cursor {
           62  +  sqlite3_vtab_cursor base;  /* Base class - must be first */
           63  +  sqlite3 *db;               /* Database connection for this cursor */
           64  +  sqlite3_stmt *pStmt;       /* Statement cursor is currently pointing at */
           65  +  sqlite3_int64 iRowid;      /* The rowid */
           66  +};
           67  +
           68  +/*
           69  +** The stmtsConnect() method is invoked to create a new
           70  +** stmts_vtab that describes the generate_stmts virtual table.
           71  +**
           72  +** Think of this routine as the constructor for stmts_vtab objects.
           73  +**
           74  +** All this routine needs to do is:
           75  +**
           76  +**    (1) Allocate the stmts_vtab object and initialize all fields.
           77  +**
           78  +**    (2) Tell SQLite (via the sqlite3_declare_vtab() interface) what the
           79  +**        result set of queries against generate_stmts will look like.
           80  +*/
           81  +static int stmtsConnect(
           82  +  sqlite3 *db,
           83  +  void *pAux,
           84  +  int argc, const char *const*argv,
           85  +  sqlite3_vtab **ppVtab,
           86  +  char **pzErr
           87  +){
           88  +  stmts_vtab *pNew;
           89  +  int rc;
           90  +
           91  +/* Column numbers */
           92  +#define STMTS_COLUMN_PTR   0    /* Numeric value of the statement pointer */
           93  +#define STMTS_COLUMN_SQL   1    /* SQL for the statement */
           94  +#define STMTS_COLUMN_NCOL  2    /* Number of result columns */
           95  +#define STMTS_COLUMN_RO    3    /* True if read-only */
           96  +#define STMTS_COLUMN_BUSY  4    /* True if currently busy */
           97  +#define STMTS_COLUMN_NSCAN 5    /* SQLITE_STMTSTATUS_FULLSCAN_STEP */
           98  +#define STMTS_COLUMN_NSORT 6    /* SQLITE_STMTSTATUS_SORT */
           99  +#define STMTS_COLUMN_NAIDX 7    /* SQLITE_STMTSTATUS_AUTOINDEX */
          100  +#define STMTS_COLUMN_NSTEP 8    /* SQLITE_STMTSTATUS_VM_STEP */
          101  +#define STMTS_COLUMN_MEM   9    /* SQLITE_STMTSTATUS_MEMUSED */
          102  +
          103  +
          104  +  rc = sqlite3_declare_vtab(db,
          105  +     "CREATE TABLE x(ptr,sql,ncol,ro,busy,nscan,nsort,naidx,nstep,mem)");
          106  +  if( rc==SQLITE_OK ){
          107  +    pNew = sqlite3_malloc( sizeof(*pNew) );
          108  +    *ppVtab = (sqlite3_vtab*)pNew;
          109  +    if( pNew==0 ) return SQLITE_NOMEM;
          110  +    memset(pNew, 0, sizeof(*pNew));
          111  +    pNew->db = db;
          112  +  }
          113  +  return rc;
          114  +}
          115  +
          116  +/*
          117  +** This method is the destructor for stmts_cursor objects.
          118  +*/
          119  +static int stmtsDisconnect(sqlite3_vtab *pVtab){
          120  +  sqlite3_free(pVtab);
          121  +  return SQLITE_OK;
          122  +}
          123  +
          124  +/*
          125  +** Constructor for a new stmts_cursor object.
          126  +*/
          127  +static int stmtsOpen(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCursor){
          128  +  stmts_cursor *pCur;
          129  +  pCur = sqlite3_malloc( sizeof(*pCur) );
          130  +  if( pCur==0 ) return SQLITE_NOMEM;
          131  +  memset(pCur, 0, sizeof(*pCur));
          132  +  pCur->db = ((stmts_vtab*)p)->db;
          133  +  *ppCursor = &pCur->base;
          134  +  return SQLITE_OK;
          135  +}
          136  +
          137  +/*
          138  +** Destructor for a stmts_cursor.
          139  +*/
          140  +static int stmtsClose(sqlite3_vtab_cursor *cur){
          141  +  sqlite3_free(cur);
          142  +  return SQLITE_OK;
          143  +}
          144  +
          145  +
          146  +/*
          147  +** Advance a stmts_cursor to its next row of output.
          148  +*/
          149  +static int stmtsNext(sqlite3_vtab_cursor *cur){
          150  +  stmts_cursor *pCur = (stmts_cursor*)cur;
          151  +  pCur->iRowid++;
          152  +  pCur->pStmt = sqlite3_next_stmt(pCur->db, pCur->pStmt);
          153  +  return SQLITE_OK;
          154  +}
          155  +
          156  +/*
          157  +** Return values of columns for the row at which the stmts_cursor
          158  +** is currently pointing.
          159  +*/
          160  +static int stmtsColumn(
          161  +  sqlite3_vtab_cursor *cur,   /* The cursor */
          162  +  sqlite3_context *ctx,       /* First argument to sqlite3_result_...() */
          163  +  int i                       /* Which column to return */
          164  +){
          165  +  stmts_cursor *pCur = (stmts_cursor*)cur;
          166  +  switch( i ){
          167  +    case STMTS_COLUMN_PTR: {
          168  +      sqlite3_result_int64(ctx, SQLITE_PTR_TO_INT64(pCur->pStmt));
          169  +      break;
          170  +    }
          171  +    case STMTS_COLUMN_SQL: {
          172  +      sqlite3_result_text(ctx, sqlite3_sql(pCur->pStmt), -1, SQLITE_TRANSIENT);
          173  +      break;
          174  +    }
          175  +    case STMTS_COLUMN_NCOL: {
          176  +      sqlite3_result_int(ctx, sqlite3_column_count(pCur->pStmt));
          177  +      break;
          178  +    }
          179  +    case STMTS_COLUMN_RO: {
          180  +      sqlite3_result_int(ctx, sqlite3_stmt_readonly(pCur->pStmt));
          181  +      break;
          182  +    }
          183  +    case STMTS_COLUMN_BUSY: {
          184  +      sqlite3_result_int(ctx, sqlite3_stmt_busy(pCur->pStmt));
          185  +      break;
          186  +    }
          187  +    case STMTS_COLUMN_NSCAN:
          188  +    case STMTS_COLUMN_NSORT:
          189  +    case STMTS_COLUMN_NAIDX:
          190  +    case STMTS_COLUMN_NSTEP:
          191  +    case STMTS_COLUMN_MEM: {
          192  +      sqlite3_result_int(ctx, sqlite3_stmt_status(pCur->pStmt,
          193  +                      i-STMTS_COLUMN_NSCAN+SQLITE_STMTSTATUS_FULLSCAN_STEP, 0));
          194  +      break;
          195  +    }
          196  +  }
          197  +  return SQLITE_OK;
          198  +}
          199  +
          200  +/*
          201  +** Return the rowid for the current row.  In this implementation, the
          202  +** rowid is the same as the output value.
          203  +*/
          204  +static int stmtsRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
          205  +  stmts_cursor *pCur = (stmts_cursor*)cur;
          206  +  *pRowid = pCur->iRowid;
          207  +  return SQLITE_OK;
          208  +}
          209  +
          210  +/*
          211  +** Return TRUE if the cursor has been moved off of the last
          212  +** row of output.
          213  +*/
          214  +static int stmtsEof(sqlite3_vtab_cursor *cur){
          215  +  stmts_cursor *pCur = (stmts_cursor*)cur;
          216  +  return pCur->pStmt==0;
          217  +}
          218  +
          219  +/*
          220  +** This method is called to "rewind" the stmts_cursor object back
          221  +** to the first row of output.  This method is always called at least
          222  +** once prior to any call to stmtsColumn() or stmtsRowid() or 
          223  +** stmtsEof().
          224  +*/
          225  +static int stmtsFilter(
          226  +  sqlite3_vtab_cursor *pVtabCursor, 
          227  +  int idxNum, const char *idxStr,
          228  +  int argc, sqlite3_value **argv
          229  +){
          230  +  stmts_cursor *pCur = (stmts_cursor *)pVtabCursor;
          231  +  pCur->pStmt = 0;
          232  +  pCur->iRowid = 0;
          233  +  return stmtsNext(pVtabCursor);
          234  +}
          235  +
          236  +/*
          237  +** SQLite will invoke this method one or more times while planning a query
          238  +** that uses the generate_stmts virtual table.  This routine needs to create
          239  +** a query plan for each invocation and compute an estimated cost for that
          240  +** plan.
          241  +*/
          242  +static int stmtsBestIndex(
          243  +  sqlite3_vtab *tab,
          244  +  sqlite3_index_info *pIdxInfo
          245  +){
          246  +  pIdxInfo->estimatedCost = (double)500;
          247  +  pIdxInfo->estimatedRows = 500;
          248  +  return SQLITE_OK;
          249  +}
          250  +
          251  +/*
          252  +** This following structure defines all the methods for the 
          253  +** generate_stmts virtual table.
          254  +*/
          255  +static sqlite3_module stmtsModule = {
          256  +  0,                         /* iVersion */
          257  +  0,                         /* xCreate */
          258  +  stmtsConnect,             /* xConnect */
          259  +  stmtsBestIndex,           /* xBestIndex */
          260  +  stmtsDisconnect,          /* xDisconnect */
          261  +  0,                         /* xDestroy */
          262  +  stmtsOpen,                /* xOpen - open a cursor */
          263  +  stmtsClose,               /* xClose - close a cursor */
          264  +  stmtsFilter,              /* xFilter - configure scan constraints */
          265  +  stmtsNext,                /* xNext - advance a cursor */
          266  +  stmtsEof,                 /* xEof - check for end of scan */
          267  +  stmtsColumn,              /* xColumn - read data */
          268  +  stmtsRowid,               /* xRowid - read data */
          269  +  0,                         /* xUpdate */
          270  +  0,                         /* xBegin */
          271  +  0,                         /* xSync */
          272  +  0,                         /* xCommit */
          273  +  0,                         /* xRollback */
          274  +  0,                         /* xFindMethod */
          275  +  0,                         /* xRename */
          276  +};
          277  +
          278  +#endif /* SQLITE_OMIT_VIRTUALTABLE */
          279  +
          280  +#ifdef _WIN32
          281  +__declspec(dllexport)
          282  +#endif
          283  +int sqlite3_stmts_init(
          284  +  sqlite3 *db, 
          285  +  char **pzErrMsg, 
          286  +  const sqlite3_api_routines *pApi
          287  +){
          288  +  int rc = SQLITE_OK;
          289  +  SQLITE_EXTENSION_INIT2(pApi);
          290  +#ifndef SQLITE_OMIT_VIRTUALTABLE
          291  +  if( sqlite3_libversion_number()<3008012 ){
          292  +    *pzErrMsg = sqlite3_mprintf(
          293  +        "generate_stmts() requires SQLite 3.8.12 or later");
          294  +    return SQLITE_ERROR;
          295  +  }
          296  +  rc = sqlite3_create_module(db, "stmts", &stmtsModule, 0);
          297  +#endif
          298  +  return rc;
          299  +}

Changes to ext/rbu/rbuvacuum.test.

   294    294     sqlite3rbu_vacuum rbu test.db state.db
   295    295     rbu step
   296    296   } {SQLITE_ERROR}
   297    297   do_test 2.1.2 {
   298    298     list [catch { rbu close } msg] $msg
   299    299   } {1 {SQLITE_ERROR - cannot vacuum wal mode database}}
   300    300   
          301  +do_test 2.1.3 {
          302  +  sqlite3rbu_vacuum rbu test.db state.db
          303  +  rbu step
          304  +} {SQLITE_ERROR}
          305  +do_test 2.1.4 {
          306  +  list [catch { rbu close_no_error } msg] $msg
          307  +} {1 SQLITE_ERROR}
          308  +
   301    309   reset_db
   302    310   do_execsql_test 2.2.0 {
   303    311     CREATE TABLE tx(a PRIMARY KEY, b BLOB);
   304    312     INSERT INTO tx VALUES(1, randomblob(900));
   305    313     INSERT INTO tx SELECT a+1, randomblob(900) FROM tx;
   306    314     INSERT INTO tx SELECT a+2, randomblob(900) FROM tx;
   307    315     INSERT INTO tx SELECT a+4, randomblob(900) FROM tx;

Changes to ext/rbu/sqlite3rbu.c.

  3783   3783       sqlite3_close(p->dbMain);
  3784   3784       rbuDeleteVfs(p);
  3785   3785       sqlite3_free(p->aBuf);
  3786   3786       sqlite3_free(p->aFrame);
  3787   3787   
  3788   3788       rbuEditErrmsg(p);
  3789   3789       rc = p->rc;
  3790         -    *pzErrmsg = p->zErrmsg;
         3790  +    if( pzErrmsg ){
         3791  +      *pzErrmsg = p->zErrmsg;
         3792  +    }else{
         3793  +      sqlite3_free(p->zErrmsg);
         3794  +    }
  3791   3795       sqlite3_free(p->zState);
  3792   3796       sqlite3_free(p);
  3793   3797     }else{
  3794   3798       rc = SQLITE_NOMEM;
  3795   3799       *pzErrmsg = 0;
  3796   3800     }
  3797   3801     return rc;

Changes to ext/rbu/sqlite3rbu.h.

   416    416   **
   417    417   ** If the RBU update has been completely applied, mark the RBU database
   418    418   ** as fully applied. Otherwise, assuming no error has occurred, save the
   419    419   ** current state of the RBU update appliation to the RBU database.
   420    420   **
   421    421   ** If an error has already occurred as part of an sqlite3rbu_step()
   422    422   ** or sqlite3rbu_open() call, or if one occurs within this function, an
   423         -** SQLite error code is returned. Additionally, *pzErrmsg may be set to
   424         -** point to a buffer containing a utf-8 formatted English language error
   425         -** message. It is the responsibility of the caller to eventually free any 
   426         -** such buffer using sqlite3_free().
          423  +** SQLite error code is returned. Additionally, if pzErrmsg is not NULL,
          424  +** *pzErrmsg may be set to point to a buffer containing a utf-8 formatted
          425  +** English language error message. It is the responsibility of the caller to
          426  +** eventually free any such buffer using sqlite3_free().
   427    427   **
   428    428   ** Otherwise, if no error occurs, this function returns SQLITE_OK if the
   429    429   ** update has been partially applied, or SQLITE_DONE if it has been 
   430    430   ** completely applied.
   431    431   */
   432    432   int sqlite3rbu_close(sqlite3rbu *pRbu, char **pzErrmsg);
   433    433   

Changes to ext/rbu/test_rbu.c.

    74     74       {"create_rbu_delta", 2, ""},  /* 2 */
    75     75       {"savestate", 2, ""},         /* 3 */
    76     76       {"dbMain_eval", 3, "SQL"},    /* 4 */
    77     77       {"bp_progress", 2, ""},       /* 5 */
    78     78       {"db", 3, "RBU"},             /* 6 */
    79     79       {"state", 2, ""},             /* 7 */
    80     80       {"progress", 2, ""},          /* 8 */
           81  +    {"close_no_error", 2, ""},    /* 9 */
    81     82       {0,0,0}
    82     83     };
    83     84     int iCmd;
    84     85   
    85     86     if( objc<2 ){
    86     87       Tcl_WrongNumArgs(interp, 1, objv, "METHOD");
    87     88       return TCL_ERROR;
................................................................................
    98     99     switch( iCmd ){
    99    100       case 0: /* step */ {
   100    101         int rc = sqlite3rbu_step(pRbu);
   101    102         Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
   102    103         break;
   103    104       }
   104    105   
          106  +    case 9: /* close_no_error */ 
   105    107       case 1: /* close */ {
   106    108         char *zErrmsg = 0;
   107    109         int rc;
   108    110         Tcl_DeleteCommand(interp, Tcl_GetString(objv[0]));
   109         -      rc = sqlite3rbu_close(pRbu, &zErrmsg);
          111  +      if( iCmd==1 ){
          112  +        rc = sqlite3rbu_close(pRbu, &zErrmsg);
          113  +      }else{
          114  +        rc = sqlite3rbu_close(pRbu, 0);
          115  +      }
   110    116         if( rc==SQLITE_OK || rc==SQLITE_DONE ){
   111    117           Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
   112    118           assert( zErrmsg==0 );
   113    119         }else{
   114    120           Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
   115    121           if( zErrmsg ){
   116    122             Tcl_AppendResult(interp, " - ", zErrmsg, 0);

Changes to ext/session/sessionat.test.

   205    205       set c2 [sql_exec_changeset db2 {
   206    206         INSERT INTO t6 VALUES(3, 3, 3, 3, 3);
   207    207         INSERT INTO t6 VALUES(4, 4, 4, 4, 4);
   208    208       }]
   209    209       list [catch { sqlite3changeset_concat $c1 $c2} msg] $msg
   210    210     } {1 SQLITE_SCHEMA}
   211    211   
          212  +  #-----------------------------------------------------------------------
          213  +  db2 close
          214  +  sqlite3 db2 test.db
          215  +  do_execsql_test $tn.6.0 {
          216  +    CREATE TABLE t7(a INTEGER PRIMARY KEY, b) %WR%;
          217  +    INSERT INTO t7 VALUES(1, 1);
          218  +    INSERT INTO t7 VALUES(2, 2);
          219  +    INSERT INTO t7 VALUES(3, 3);
          220  +  }
          221  +
          222  +  do_test $tn.6.1 {
          223  +    set c1 [sql_exec_changeset db {
          224  +      INSERT INTO t7 VALUES(4, 4);
          225  +      DELETE FROM t7 WHERE a=1;
          226  +      UPDATE t7 SET b=222 WHERE a=2;
          227  +    }]
          228  +    set cinv [sqlite3changeset_invert $c1]
          229  +    execsql { SELECT * FROM t7 }
          230  +  } {2 222 3 3 4 4}
          231  +
          232  +  do_execsql_test -db db2 $tn.6.2 {
          233  +    ALTER TABLE t7 ADD COLUMN c DEFAULT 'ccc'
          234  +  }
          235  +
          236  +  proc xConfict {args} { return "OMIT" }
          237  +  do_test $tn.6.3 {
          238  +    sqlite3changeset_apply db $cinv xConflict
          239  +    execsql { SELECT * FROM t7 }
          240  +  } {1 1 ccc 2 2 ccc 3 3 ccc}
   212    241   }]
   213    242   }
   214    243   
   215    244   
   216    245   finish_test

Added ext/session/sessiondiff.test.

            1  +# 2015-07-31
            2  +#
            3  +# The author disclaims copyright to this source code.  In place of
            4  +# a legal notice, here is a blessing:
            5  +#
            6  +#    May you do good and not evil.
            7  +#    May you find forgiveness for yourself and forgive others.
            8  +#    May you share freely, never taking more than you give.
            9  +#
           10  +#***********************************************************************
           11  +#
           12  +# Tests for the [sqldiff --changeset] command.
           13  +#
           14  +#
           15  +if {![info exists testdir]} {
           16  +  set testdir [file join [file dirname [info script]] .. .. test]
           17  +}
           18  +source $testdir/tester.tcl
           19  +ifcapable !session {finish_test; return}
           20  +set testprefix sessiondiff
           21  +
           22  +set PROG [test_find_sqldiff]
           23  +db close
           24  +
           25  +proc sqlesc {id} {
           26  +  set ret "'[string map {' ''} $id]'"
           27  +  set ret
           28  +}
           29  +
           30  +proc database_cksum {db1} {
           31  +  set txt ""
           32  +
           33  +  sqlite3 dbtmp $db1
           34  +  foreach tbl [dbtmp eval {SELECT name FROM sqlite_master WHERE type='table'}] {
           35  +    set cols [list]
           36  +    dbtmp eval "PRAGMA table_info = [sqlesc $tbl]" { 
           37  +      lappend cols "quote( $name )" 
           38  +    }
           39  +    append txt [dbtmp eval \
           40  +      "SELECT [join $cols {||'.'||}] FROM [sqlesc $tbl] ORDER BY 1"
           41  +    ]
           42  +  }
           43  +  dbtmp close
           44  +
           45  +  md5 $txt
           46  +}
           47  +
           48  +proc readfile {filename} {
           49  +  set fd [open $filename]
           50  +  fconfigure $fd -translation binary -encoding binary
           51  +  set data [read $fd]
           52  +  close $fd
           53  +  set data
           54  +}
           55  +
           56  +proc get_changeset {db1 db2} {
           57  +  exec $::PROG --changeset changeset.bin $db1 $db2
           58  +  set bin [readfile changeset.bin]
           59  +  return $bin
           60  +}
           61  +
           62  +proc xConflict {args} { 
           63  +  return "" 
           64  +}
           65  +
           66  +proc do_changeset_test {tn sql1 sql2} {
           67  +  forcedelete test.db123 test.db124 
           68  +
           69  +  sqlite3 db test.db123
           70  +  db eval $sql1
           71  +  db close
           72  +
           73  +  sqlite3 db test.db124
           74  +  db eval $sql2
           75  +
           76  +  set cs [get_changeset test.db124 test.db123]
           77  +  sqlite3changeset_apply db $cs xConflict
           78  +  db close
           79  +
           80  +  set database_cksum1 [database_cksum test.db123]
           81  +  set database_cksum2 [database_cksum test.db124]
           82  +
           83  +  uplevel [list \
           84  +      do_test $tn [list string compare $database_cksum1 $database_cksum2] 0
           85  +  ]
           86  +}
           87  +
           88  +do_changeset_test 1.0 {
           89  +  CREATE TABLE t1(x PRIMARY KEY);
           90  +} {
           91  +  CREATE TABLE t1(x PRIMARY KEY);
           92  +}
           93  +
           94  +do_changeset_test 1.1 {
           95  +  CREATE TABLE t1(x PRIMARY KEY);
           96  +  CREATE TABLE t2(x PRIMARY KEY, y);
           97  +  INSERT INTO t2 VALUES(1, 2);
           98  +} {
           99  +  CREATE TABLE t1(x PRIMARY KEY);
          100  +  CREATE TABLE t2(x PRIMARY KEY, y);
          101  +  INSERT INTO t2 VALUES(3, 4);
          102  +}
          103  +
          104  +do_changeset_test 1.2 {
          105  +  CREATE TABLE t2(a, b, c, PRIMARY KEY(b, c));
          106  +  INSERT INTO t2 VALUES(1, 2, 3);
          107  +  INSERT INTO t2 VALUES(4, 5, 6);
          108  +} {
          109  +  CREATE TABLE t2(a, b, c, PRIMARY KEY(b, c));
          110  +  INSERT INTO t2 VALUES(1, 2, 11);
          111  +  INSERT INTO t2 VALUES(7, 8, 9);
          112  +}
          113  +
          114  +finish_test

Changes to ext/session/sqlite3session.c.

  2832   2832       return SQLITE_DONE;
  2833   2833     }
  2834   2834   
  2835   2835     sessionDiscardData(&p->in);
  2836   2836     p->in.iCurrent = p->in.iNext;
  2837   2837   
  2838   2838     op = p->in.aData[p->in.iNext++];
  2839         -  if( op=='T' || op=='P' ){
         2839  +  while( op=='T' || op=='P' ){
  2840   2840       p->bPatchset = (op=='P');
  2841   2841       if( sessionChangesetReadTblhdr(p) ) return p->rc;
  2842   2842       if( (p->rc = sessionInputBuffer(&p->in, 2)) ) return p->rc;
  2843   2843       p->in.iCurrent = p->in.iNext;
         2844  +    if( p->in.iNext>=p->in.nData ) return SQLITE_DONE;
  2844   2845       op = p->in.aData[p->in.iNext++];
  2845   2846     }
  2846   2847   
  2847   2848     p->op = op;
  2848   2849     p->bIndirect = p->in.aData[p->in.iNext++];
  2849   2850     if( p->op!=SQLITE_UPDATE && p->op!=SQLITE_DELETE && p->op!=SQLITE_INSERT ){
  2850   2851       return (p->rc = SQLITE_CORRUPT_BKPT);

Changes to src/attach.c.

    65     65     int rc = 0;
    66     66     sqlite3 *db = sqlite3_context_db_handle(context);
    67     67     const char *zName;
    68     68     const char *zFile;
    69     69     char *zPath = 0;
    70     70     char *zErr = 0;
    71     71     unsigned int flags;
    72         -  Db *aNew;
           72  +  Db *aNew;                 /* New array of Db pointers */
           73  +  Db *pNew;                 /* Db object for the newly attached database */
    73     74     char *zErrDyn = 0;
    74     75     sqlite3_vfs *pVfs;
    75     76   
    76     77     UNUSED_PARAMETER(NotUsed);
    77     78   
    78     79     zFile = (const char *)sqlite3_value_text(argv[0]);
    79     80     zName = (const char *)sqlite3_value_text(argv[1]);
................................................................................
   113    114       if( aNew==0 ) return;
   114    115       memcpy(aNew, db->aDb, sizeof(db->aDb[0])*2);
   115    116     }else{
   116    117       aNew = sqlite3DbRealloc(db, db->aDb, sizeof(db->aDb[0])*(db->nDb+1) );
   117    118       if( aNew==0 ) return;
   118    119     }
   119    120     db->aDb = aNew;
   120         -  aNew = &db->aDb[db->nDb];
   121         -  memset(aNew, 0, sizeof(*aNew));
          121  +  pNew = &db->aDb[db->nDb];
          122  +  memset(pNew, 0, sizeof(*pNew));
   122    123   
   123    124     /* Open the database file. If the btree is successfully opened, use
   124    125     ** it to obtain the database schema. At this point the schema may
   125    126     ** or may not be initialized.
   126    127     */
   127    128     flags = db->openFlags;
   128    129     rc = sqlite3ParseUri(db->pVfs->zName, zFile, &flags, &pVfs, &zPath, &zErr);
................................................................................
   130    131       if( rc==SQLITE_NOMEM ) sqlite3OomFault(db);
   131    132       sqlite3_result_error(context, zErr, -1);
   132    133       sqlite3_free(zErr);
   133    134       return;
   134    135     }
   135    136     assert( pVfs );
   136    137     flags |= SQLITE_OPEN_MAIN_DB;
   137         -  rc = sqlite3BtreeOpen(pVfs, zPath, db, &aNew->pBt, 0, flags);
          138  +  rc = sqlite3BtreeOpen(pVfs, zPath, db, &pNew->pBt, 0, flags);
   138    139     sqlite3_free( zPath );
   139    140     db->nDb++;
   140    141     db->skipBtreeMutex = 0;
   141    142     if( rc==SQLITE_CONSTRAINT ){
   142    143       rc = SQLITE_ERROR;
   143    144       zErrDyn = sqlite3MPrintf(db, "database is already attached");
   144    145     }else if( rc==SQLITE_OK ){
   145    146       Pager *pPager;
   146         -    aNew->pSchema = sqlite3SchemaGet(db, aNew->pBt);
   147         -    if( !aNew->pSchema ){
          147  +    pNew->pSchema = sqlite3SchemaGet(db, pNew->pBt);
          148  +    if( !pNew->pSchema ){
   148    149         rc = SQLITE_NOMEM_BKPT;
   149         -    }else if( aNew->pSchema->file_format && aNew->pSchema->enc!=ENC(db) ){
          150  +    }else if( pNew->pSchema->file_format && pNew->pSchema->enc!=ENC(db) ){
   150    151         zErrDyn = sqlite3MPrintf(db, 
   151    152           "attached databases must use the same text encoding as main database");
   152    153         rc = SQLITE_ERROR;
   153    154       }
   154         -    sqlite3BtreeEnter(aNew->pBt);
   155         -    pPager = sqlite3BtreePager(aNew->pBt);
          155  +    sqlite3BtreeEnter(pNew->pBt);
          156  +    pPager = sqlite3BtreePager(pNew->pBt);
   156    157       sqlite3PagerLockingMode(pPager, db->dfltLockMode);
   157         -    sqlite3BtreeSecureDelete(aNew->pBt,
          158  +    sqlite3BtreeSecureDelete(pNew->pBt,
   158    159                                sqlite3BtreeSecureDelete(db->aDb[0].pBt,-1) );
   159    160   #ifndef SQLITE_OMIT_PAGER_PRAGMAS
   160         -    sqlite3BtreeSetPagerFlags(aNew->pBt,
          161  +    sqlite3BtreeSetPagerFlags(pNew->pBt,
   161    162                         PAGER_SYNCHRONOUS_FULL | (db->flags & PAGER_FLAGS_MASK));
   162    163   #endif
   163         -    sqlite3BtreeLeave(aNew->pBt);
          164  +    sqlite3BtreeLeave(pNew->pBt);
   164    165     }
   165         -  aNew->safety_level = SQLITE_DEFAULT_SYNCHRONOUS+1;
   166         -  aNew->zDbSName = sqlite3DbStrDup(db, zName);
   167         -  if( rc==SQLITE_OK && aNew->zDbSName==0 ){
          166  +  pNew->safety_level = SQLITE_DEFAULT_SYNCHRONOUS+1;
          167  +  pNew->zDbSName = sqlite3DbStrDup(db, zName);
          168  +  if( rc==SQLITE_OK && pNew->zDbSName==0 ){
   168    169       rc = SQLITE_NOMEM_BKPT;
   169    170     }
   170    171   
   171    172   
   172    173   #ifdef SQLITE_HAS_CODEC
   173    174     if( rc==SQLITE_OK ){
   174    175       extern int sqlite3CodecAttach(sqlite3*, int, const void*, int);

Changes to src/btree.c.

   773    773   
   774    774     if( pKey ){
   775    775       assert( nKey==(i64)(int)nKey );
   776    776       pIdxKey = sqlite3VdbeAllocUnpackedRecord(pCur->pKeyInfo);
   777    777       if( pIdxKey==0 ) return SQLITE_NOMEM_BKPT;
   778    778       sqlite3VdbeRecordUnpack(pCur->pKeyInfo, (int)nKey, pKey, pIdxKey);
   779    779       if( pIdxKey->nField==0 ){
   780         -      rc = SQLITE_CORRUPT_BKPT;
          780  +      rc = SQLITE_CORRUPT_PGNO(pCur->apPage[pCur->iPage]->pgno);
   781    781         goto moveto_done;
   782    782       }
   783    783     }else{
   784    784       pIdxKey = 0;
   785    785     }
   786    786     rc = sqlite3BtreeMovetoUnpacked(pCur, pIdxKey, nKey, bias, pRes);
   787    787   moveto_done:
................................................................................
  1002   1002     }
  1003   1003     assert( offset <= (int)pBt->usableSize-5 );
  1004   1004     assert( pEType!=0 );
  1005   1005     *pEType = pPtrmap[offset];
  1006   1006     if( pPgno ) *pPgno = get4byte(&pPtrmap[offset+1]);
  1007   1007   
  1008   1008     sqlite3PagerUnref(pDbPage);
  1009         -  if( *pEType<1 || *pEType>5 ) return SQLITE_CORRUPT_BKPT;
         1009  +  if( *pEType<1 || *pEType>5 ) return SQLITE_CORRUPT_PGNO(iPtrmap);
  1010   1010     return SQLITE_OK;
  1011   1011   }
  1012   1012   
  1013   1013   #else /* if defined SQLITE_OMIT_AUTOVACUUM */
  1014   1014     #define ptrmapPut(w,x,y,z,rc)
  1015   1015     #define ptrmapGet(w,x,y,z) SQLITE_OK
  1016   1016     #define ptrmapPutOvflPtr(x, y, rc)
................................................................................
  1387   1387         if( 0==iFree2 || (data[iFree2]==0 && data[iFree2+1]==0) ){
  1388   1388           u8 *pEnd = &data[cellOffset + nCell*2];
  1389   1389           u8 *pAddr;
  1390   1390           int sz2 = 0;
  1391   1391           int sz = get2byte(&data[iFree+2]);
  1392   1392           int top = get2byte(&data[hdr+5]);
  1393   1393           if( iFree2 ){
  1394         -          if( iFree+sz>iFree2 ) return SQLITE_CORRUPT_BKPT;
         1394  +          if( iFree+sz>iFree2 ) return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1395   1395             sz2 = get2byte(&data[iFree2+2]);
  1396   1396             assert( iFree+sz+sz2+iFree2-(iFree+sz) <= usableSize );
  1397   1397             memmove(&data[iFree+sz+sz2], &data[iFree+sz], iFree2-(iFree+sz));
  1398   1398             sz += sz2;
  1399   1399           }
  1400   1400           cbrk = top+sz;
  1401   1401           assert( cbrk+(iFree-top) <= usableSize );
................................................................................
  1418   1418       pc = get2byte(pAddr);
  1419   1419       testcase( pc==iCellFirst );
  1420   1420       testcase( pc==iCellLast );
  1421   1421       /* These conditions have already been verified in btreeInitPage()
  1422   1422       ** if PRAGMA cell_size_check=ON.
  1423   1423       */
  1424   1424       if( pc<iCellFirst || pc>iCellLast ){
  1425         -      return SQLITE_CORRUPT_BKPT;
         1425  +      return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1426   1426       }
  1427   1427       assert( pc>=iCellFirst && pc<=iCellLast );
  1428   1428       size = pPage->xCellSize(pPage, &src[pc]);
  1429   1429       cbrk -= size;
  1430   1430       if( cbrk<iCellFirst || pc+size>usableSize ){
  1431         -      return SQLITE_CORRUPT_BKPT;
         1431  +      return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1432   1432       }
  1433   1433       assert( cbrk+size<=usableSize && cbrk>=iCellFirst );
  1434   1434       testcase( cbrk+size==usableSize );
  1435   1435       testcase( pc+size==usableSize );
  1436   1436       put2byte(pAddr, cbrk);
  1437   1437       if( temp==0 ){
  1438   1438         int x;
................................................................................
  1444   1444       }
  1445   1445       memcpy(&data[cbrk], &src[pc], size);
  1446   1446     }
  1447   1447     data[hdr+7] = 0;
  1448   1448   
  1449   1449    defragment_out:
  1450   1450     if( data[hdr+7]+cbrk-iCellFirst!=pPage->nFree ){
  1451         -    return SQLITE_CORRUPT_BKPT;
         1451  +    return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1452   1452     }
  1453   1453     assert( cbrk>=iCellFirst );
  1454   1454     put2byte(&data[hdr+5], cbrk);
  1455   1455     data[hdr+1] = 0;
  1456   1456     data[hdr+2] = 0;
  1457   1457     memset(&data[iCellFirst], 0, cbrk-iCellFirst);
  1458   1458     assert( sqlite3PagerIswriteable(pPage->pDbPage) );
................................................................................
  1483   1483   
  1484   1484     assert( pc>0 );
  1485   1485     do{
  1486   1486       int size;            /* Size of the free slot */
  1487   1487       /* EVIDENCE-OF: R-06866-39125 Freeblocks are always connected in order of
  1488   1488       ** increasing offset. */
  1489   1489       if( pc>usableSize-4 || pc<iAddr+4 ){
  1490         -      *pRc = SQLITE_CORRUPT_BKPT;
         1490  +      *pRc = SQLITE_CORRUPT_PGNO(pPg->pgno);
  1491   1491         return 0;
  1492   1492       }
  1493   1493       /* EVIDENCE-OF: R-22710-53328 The third and fourth bytes of each
  1494   1494       ** freeblock form a big-endian integer which is the size of the freeblock
  1495   1495       ** in bytes, including the 4-byte header. */
  1496   1496       size = get2byte(&aData[pc+2]);
  1497   1497       if( (x = size - nByte)>=0 ){
  1498   1498         testcase( x==4 );
  1499   1499         testcase( x==3 );
  1500   1500         if( pc < pPg->cellOffset+2*pPg->nCell || size+pc > usableSize ){
  1501         -        *pRc = SQLITE_CORRUPT_BKPT;
         1501  +        *pRc = SQLITE_CORRUPT_PGNO(pPg->pgno);
  1502   1502           return 0;
  1503   1503         }else if( x<4 ){
  1504   1504           /* EVIDENCE-OF: R-11498-58022 In a well-formed b-tree page, the total
  1505   1505           ** number of bytes in fragments may not exceed 60. */
  1506   1506           if( aData[hdr+7]>57 ) return 0;
  1507   1507   
  1508   1508           /* Remove the slot from the free-list. Update the number of
................................................................................
  1561   1561     ** integer, so a value of 0 is used in its place. */
  1562   1562     top = get2byte(&data[hdr+5]);
  1563   1563     assert( top<=(int)pPage->pBt->usableSize ); /* Prevent by getAndInitPage() */
  1564   1564     if( gap>top ){
  1565   1565       if( top==0 && pPage->pBt->usableSize==65536 ){
  1566   1566         top = 65536;
  1567   1567       }else{
  1568         -      return SQLITE_CORRUPT_BKPT;
         1568  +      return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1569   1569       }
  1570   1570     }
  1571   1571   
  1572   1572     /* If there is enough space between gap and top for one more cell pointer
  1573   1573     ** array entry offset, and if the freelist is not empty, then search the
  1574   1574     ** freelist looking for a free slot big enough to satisfy the request.
  1575   1575     */
................................................................................
  1657   1657     iPtr = hdr + 1;
  1658   1658     if( data[iPtr+1]==0 && data[iPtr]==0 ){
  1659   1659       iFreeBlk = 0;  /* Shortcut for the case when the freelist is empty */
  1660   1660     }else{
  1661   1661       while( (iFreeBlk = get2byte(&data[iPtr]))<iStart ){
  1662   1662         if( iFreeBlk<iPtr+4 ){
  1663   1663           if( iFreeBlk==0 ) break;
  1664         -        return SQLITE_CORRUPT_BKPT;
         1664  +        return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1665   1665         }
  1666   1666         iPtr = iFreeBlk;
  1667   1667       }
  1668         -    if( iFreeBlk>iLast ) return SQLITE_CORRUPT_BKPT;
         1668  +    if( iFreeBlk>iLast ) return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1669   1669       assert( iFreeBlk>iPtr || iFreeBlk==0 );
  1670   1670     
  1671   1671       /* At this point:
  1672   1672       **    iFreeBlk:   First freeblock after iStart, or zero if none
  1673   1673       **    iPtr:       The address of a pointer to iFreeBlk
  1674   1674       **
  1675   1675       ** Check to see if iFreeBlk should be coalesced onto the end of iStart.
  1676   1676       */
  1677   1677       if( iFreeBlk && iEnd+3>=iFreeBlk ){
  1678   1678         nFrag = iFreeBlk - iEnd;
  1679         -      if( iEnd>iFreeBlk ) return SQLITE_CORRUPT_BKPT;
         1679  +      if( iEnd>iFreeBlk ) return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1680   1680         iEnd = iFreeBlk + get2byte(&data[iFreeBlk+2]);
  1681         -      if( iEnd > pPage->pBt->usableSize ) return SQLITE_CORRUPT_BKPT;
         1681  +      if( iEnd > pPage->pBt->usableSize ){
         1682  +        return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1683  +      }
  1682   1684         iSize = iEnd - iStart;
  1683   1685         iFreeBlk = get2byte(&data[iFreeBlk]);
  1684   1686       }
  1685   1687     
  1686   1688       /* If iPtr is another freeblock (that is, if iPtr is not the freelist
  1687   1689       ** pointer in the page header) then check to see if iStart should be
  1688   1690       ** coalesced onto the end of iPtr.
  1689   1691       */
  1690   1692       if( iPtr>hdr+1 ){
  1691   1693         int iPtrEnd = iPtr + get2byte(&data[iPtr+2]);
  1692   1694         if( iPtrEnd+3>=iStart ){
  1693         -        if( iPtrEnd>iStart ) return SQLITE_CORRUPT_BKPT;
         1695  +        if( iPtrEnd>iStart ) return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1694   1696           nFrag += iStart - iPtrEnd;
  1695   1697           iSize = iEnd - iPtr;
  1696   1698           iStart = iPtr;
  1697   1699         }
  1698   1700       }
  1699         -    if( nFrag>data[hdr+7] ) return SQLITE_CORRUPT_BKPT;
         1701  +    if( nFrag>data[hdr+7] ) return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1700   1702       data[hdr+7] -= nFrag;
  1701   1703     }
  1702   1704     if( iStart==get2byte(&data[hdr+5]) ){
  1703   1705       /* The new freeblock is at the beginning of the cell content area,
  1704   1706       ** so just extend the cell content area rather than create another
  1705   1707       ** freelist entry */
  1706         -    if( iPtr!=hdr+1 ) return SQLITE_CORRUPT_BKPT;
         1708  +    if( iPtr!=hdr+1 ) return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1707   1709       put2byte(&data[hdr+1], iFreeBlk);
  1708   1710       put2byte(&data[hdr+5], iEnd);
  1709   1711     }else{
  1710   1712       /* Insert the new freeblock into the freelist */
  1711   1713       put2byte(&data[iPtr], iStart);
  1712   1714       put2byte(&data[iStart], iFreeBlk);
  1713   1715       put2byte(&data[iStart+2], iSize);
................................................................................
  1767   1769       pPage->intKeyLeaf = 0;
  1768   1770       pPage->xParseCell = btreeParseCellPtrIndex;
  1769   1771       pPage->maxLocal = pBt->maxLocal;
  1770   1772       pPage->minLocal = pBt->minLocal;
  1771   1773     }else{
  1772   1774       /* EVIDENCE-OF: R-47608-56469 Any other value for the b-tree page type is
  1773   1775       ** an error. */
  1774         -    return SQLITE_CORRUPT_BKPT;
         1776  +    return SQLITE_CORRUPT_PGNO(pPage->pgno);
  1775   1777     }
  1776   1778     pPage->max1bytePayload = pBt->max1bytePayload;
  1777   1779     return SQLITE_OK;
  1778   1780   }
  1779   1781   
  1780   1782   /*
  1781   1783   ** Initialize the auxiliary information for a disk block.
................................................................................
  1783   1785   ** Return SQLITE_OK on success.  If we see that the page does
  1784   1786   ** not contain a well-formed database page, then return 
  1785   1787   ** SQLITE_CORRUPT.  Note that a return of SQLITE_OK does not
  1786   1788   ** guarantee that the page is well-formed.  It only shows that
  1787   1789   ** we failed to detect any corruption.
  1788   1790   */
  1789   1791   static int btreeInitPage(MemPage *pPage){
         1792  +  int pc;            /* Address of a freeblock within pPage->aData[] */
         1793  +  u8 hdr;            /* Offset to beginning of page header */
         1794  +  u8 *data;          /* Equal to pPage->aData */
         1795  +  BtShared *pBt;        /* The main btree structure */
         1796  +  int usableSize;    /* Amount of usable space on each page */
         1797  +  u16 cellOffset;    /* Offset from start of page to first cell pointer */
         1798  +  int nFree;         /* Number of unused bytes on the page */
         1799  +  int top;           /* First byte of the cell content area */
         1800  +  int iCellFirst;    /* First allowable cell or freeblock offset */
         1801  +  int iCellLast;     /* Last possible cell or freeblock offset */
  1790   1802   
  1791   1803     assert( pPage->pBt!=0 );
  1792   1804     assert( pPage->pBt->db!=0 );
  1793   1805     assert( sqlite3_mutex_held(pPage->pBt->mutex) );
  1794   1806     assert( pPage->pgno==sqlite3PagerPagenumber(pPage->pDbPage) );
  1795   1807     assert( pPage == sqlite3PagerGetExtra(pPage->pDbPage) );
  1796   1808     assert( pPage->aData == sqlite3PagerGetData(pPage->pDbPage) );
  1797         -
  1798         -  if( !pPage->isInit ){
  1799         -    int pc;            /* Address of a freeblock within pPage->aData[] */
  1800         -    u8 hdr;            /* Offset to beginning of page header */
  1801         -    u8 *data;          /* Equal to pPage->aData */
  1802         -    BtShared *pBt;        /* The main btree structure */
  1803         -    int usableSize;    /* Amount of usable space on each page */
  1804         -    u16 cellOffset;    /* Offset from start of page to first cell pointer */
  1805         -    int nFree;         /* Number of unused bytes on the page */
  1806         -    int top;           /* First byte of the cell content area */
  1807         -    int iCellFirst;    /* First allowable cell or freeblock offset */
  1808         -    int iCellLast;     /* Last possible cell or freeblock offset */
  1809         -
  1810         -    pBt = pPage->pBt;
  1811         -
  1812         -    hdr = pPage->hdrOffset;
  1813         -    data = pPage->aData;
  1814         -    /* EVIDENCE-OF: R-28594-02890 The one-byte flag at offset 0 indicating
  1815         -    ** the b-tree page type. */
  1816         -    if( decodeFlags(pPage, data[hdr]) ) return SQLITE_CORRUPT_BKPT;
  1817         -    assert( pBt->pageSize>=512 && pBt->pageSize<=65536 );
  1818         -    pPage->maskPage = (u16)(pBt->pageSize - 1);
  1819         -    pPage->nOverflow = 0;
  1820         -    usableSize = pBt->usableSize;
  1821         -    pPage->cellOffset = cellOffset = hdr + 8 + pPage->childPtrSize;
  1822         -    pPage->aDataEnd = &data[usableSize];
  1823         -    pPage->aCellIdx = &data[cellOffset];
  1824         -    pPage->aDataOfst = &data[pPage->childPtrSize];
  1825         -    /* EVIDENCE-OF: R-58015-48175 The two-byte integer at offset 5 designates
  1826         -    ** the start of the cell content area. A zero value for this integer is
  1827         -    ** interpreted as 65536. */
  1828         -    top = get2byteNotZero(&data[hdr+5]);
  1829         -    /* EVIDENCE-OF: R-37002-32774 The two-byte integer at offset 3 gives the
  1830         -    ** number of cells on the page. */
  1831         -    pPage->nCell = get2byte(&data[hdr+3]);
  1832         -    if( pPage->nCell>MX_CELL(pBt) ){
  1833         -      /* To many cells for a single page.  The page must be corrupt */
  1834         -      return SQLITE_CORRUPT_BKPT;
  1835         -    }
  1836         -    testcase( pPage->nCell==MX_CELL(pBt) );
  1837         -    /* EVIDENCE-OF: R-24089-57979 If a page contains no cells (which is only
  1838         -    ** possible for a root page of a table that contains no rows) then the
  1839         -    ** offset to the cell content area will equal the page size minus the
  1840         -    ** bytes of reserved space. */
  1841         -    assert( pPage->nCell>0 || top==usableSize || CORRUPT_DB );
  1842         -
  1843         -    /* A malformed database page might cause us to read past the end
  1844         -    ** of page when parsing a cell.  
  1845         -    **
  1846         -    ** The following block of code checks early to see if a cell extends
  1847         -    ** past the end of a page boundary and causes SQLITE_CORRUPT to be 
  1848         -    ** returned if it does.
  1849         -    */
  1850         -    iCellFirst = cellOffset + 2*pPage->nCell;
  1851         -    iCellLast = usableSize - 4;
  1852         -    if( pBt->db->flags & SQLITE_CellSizeCk ){
  1853         -      int i;            /* Index into the cell pointer array */
  1854         -      int sz;           /* Size of a cell */
  1855         -
  1856         -      if( !pPage->leaf ) iCellLast--;
  1857         -      for(i=0; i<pPage->nCell; i++){
  1858         -        pc = get2byteAligned(&data[cellOffset+i*2]);
  1859         -        testcase( pc==iCellFirst );
  1860         -        testcase( pc==iCellLast );
  1861         -        if( pc<iCellFirst || pc>iCellLast ){
  1862         -          return SQLITE_CORRUPT_BKPT;
  1863         -        }
  1864         -        sz = pPage->xCellSize(pPage, &data[pc]);
  1865         -        testcase( pc+sz==usableSize );
  1866         -        if( pc+sz>usableSize ){
  1867         -          return SQLITE_CORRUPT_BKPT;
  1868         -        }
  1869         -      }
  1870         -      if( !pPage->leaf ) iCellLast++;
  1871         -    }  
  1872         -
  1873         -    /* Compute the total free space on the page
  1874         -    ** EVIDENCE-OF: R-23588-34450 The two-byte integer at offset 1 gives the
  1875         -    ** start of the first freeblock on the page, or is zero if there are no
  1876         -    ** freeblocks. */
  1877         -    pc = get2byte(&data[hdr+1]);
  1878         -    nFree = data[hdr+7] + top;  /* Init nFree to non-freeblock free space */
  1879         -    if( pc>0 ){
  1880         -      u32 next, size;
  1881         -      if( pc<iCellFirst ){
  1882         -        /* EVIDENCE-OF: R-55530-52930 In a well-formed b-tree page, there will
  1883         -        ** always be at least one cell before the first freeblock.
  1884         -        */
  1885         -        return SQLITE_CORRUPT_BKPT; 
  1886         -      }
  1887         -      while( 1 ){
  1888         -        if( pc>iCellLast ){
  1889         -          return SQLITE_CORRUPT_BKPT; /* Freeblock off the end of the page */
  1890         -        }
  1891         -        next = get2byte(&data[pc]);
  1892         -        size = get2byte(&data[pc+2]);
  1893         -        nFree = nFree + size;
  1894         -        if( next<=pc+size+3 ) break;
  1895         -        pc = next;
  1896         -      }
  1897         -      if( next>0 ){
  1898         -        return SQLITE_CORRUPT_BKPT;  /* Freeblock not in ascending order */
  1899         -      }
  1900         -      if( pc+size>(unsigned int)usableSize ){
  1901         -        return SQLITE_CORRUPT_BKPT;  /* Last freeblock extends past page end */
  1902         -      }
  1903         -    }
  1904         -
  1905         -    /* At this point, nFree contains the sum of the offset to the start
  1906         -    ** of the cell-content area plus the number of free bytes within
  1907         -    ** the cell-content area. If this is greater than the usable-size
  1908         -    ** of the page, then the page must be corrupted. This check also
  1909         -    ** serves to verify that the offset to the start of the cell-content
  1910         -    ** area, according to the page header, lies within the page.
  1911         -    */
  1912         -    if( nFree>usableSize ){
  1913         -      return SQLITE_CORRUPT_BKPT; 
  1914         -    }
  1915         -    pPage->nFree = (u16)(nFree - iCellFirst);
  1916         -    pPage->isInit = 1;
  1917         -  }
         1809  +  assert( pPage->isInit==0 );
         1810  +
         1811  +  pBt = pPage->pBt;
         1812  +  hdr = pPage->hdrOffset;
         1813  +  data = pPage->aData;
         1814  +  /* EVIDENCE-OF: R-28594-02890 The one-byte flag at offset 0 indicating
         1815  +  ** the b-tree page type. */
         1816  +  if( decodeFlags(pPage, data[hdr]) ){
         1817  +    return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1818  +  }
         1819  +  assert( pBt->pageSize>=512 && pBt->pageSize<=65536 );
         1820  +  pPage->maskPage = (u16)(pBt->pageSize - 1);
         1821  +  pPage->nOverflow = 0;
         1822  +  usableSize = pBt->usableSize;
         1823  +  pPage->cellOffset = cellOffset = hdr + 8 + pPage->childPtrSize;
         1824  +  pPage->aDataEnd = &data[usableSize];
         1825  +  pPage->aCellIdx = &data[cellOffset];
         1826  +  pPage->aDataOfst = &data[pPage->childPtrSize];
         1827  +  /* EVIDENCE-OF: R-58015-48175 The two-byte integer at offset 5 designates
         1828  +  ** the start of the cell content area. A zero value for this integer is
         1829  +  ** interpreted as 65536. */
         1830  +  top = get2byteNotZero(&data[hdr+5]);
         1831  +  /* EVIDENCE-OF: R-37002-32774 The two-byte integer at offset 3 gives the
         1832  +  ** number of cells on the page. */
         1833  +  pPage->nCell = get2byte(&data[hdr+3]);
         1834  +  if( pPage->nCell>MX_CELL(pBt) ){
         1835  +    /* To many cells for a single page.  The page must be corrupt */
         1836  +    return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1837  +  }
         1838  +  testcase( pPage->nCell==MX_CELL(pBt) );
         1839  +  /* EVIDENCE-OF: R-24089-57979 If a page contains no cells (which is only
         1840  +  ** possible for a root page of a table that contains no rows) then the
         1841  +  ** offset to the cell content area will equal the page size minus the
         1842  +  ** bytes of reserved space. */
         1843  +  assert( pPage->nCell>0 || top==usableSize || CORRUPT_DB );
         1844  +
         1845  +  /* A malformed database page might cause us to read past the end
         1846  +  ** of page when parsing a cell.  
         1847  +  **
         1848  +  ** The following block of code checks early to see if a cell extends
         1849  +  ** past the end of a page boundary and causes SQLITE_CORRUPT to be 
         1850  +  ** returned if it does.
         1851  +  */
         1852  +  iCellFirst = cellOffset + 2*pPage->nCell;
         1853  +  iCellLast = usableSize - 4;
         1854  +  if( pBt->db->flags & SQLITE_CellSizeCk ){
         1855  +    int i;            /* Index into the cell pointer array */
         1856  +    int sz;           /* Size of a cell */
         1857  +
         1858  +    if( !pPage->leaf ) iCellLast--;
         1859  +    for(i=0; i<pPage->nCell; i++){
         1860  +      pc = get2byteAligned(&data[cellOffset+i*2]);
         1861  +      testcase( pc==iCellFirst );
         1862  +      testcase( pc==iCellLast );
         1863  +      if( pc<iCellFirst || pc>iCellLast ){
         1864  +        return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1865  +      }
         1866  +      sz = pPage->xCellSize(pPage, &data[pc]);
         1867  +      testcase( pc+sz==usableSize );
         1868  +      if( pc+sz>usableSize ){
         1869  +        return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1870  +      }
         1871  +    }
         1872  +    if( !pPage->leaf ) iCellLast++;
         1873  +  }  
         1874  +
         1875  +  /* Compute the total free space on the page
         1876  +  ** EVIDENCE-OF: R-23588-34450 The two-byte integer at offset 1 gives the
         1877  +  ** start of the first freeblock on the page, or is zero if there are no
         1878  +  ** freeblocks. */
         1879  +  pc = get2byte(&data[hdr+1]);
         1880  +  nFree = data[hdr+7] + top;  /* Init nFree to non-freeblock free space */
         1881  +  if( pc>0 ){
         1882  +    u32 next, size;
         1883  +    if( pc<iCellFirst ){
         1884  +      /* EVIDENCE-OF: R-55530-52930 In a well-formed b-tree page, there will
         1885  +      ** always be at least one cell before the first freeblock.
         1886  +      */
         1887  +      return SQLITE_CORRUPT_PGNO(pPage->pgno); 
         1888  +    }
         1889  +    while( 1 ){
         1890  +      if( pc>iCellLast ){
         1891  +        /* Freeblock off the end of the page */
         1892  +        return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1893  +      }
         1894  +      next = get2byte(&data[pc]);
         1895  +      size = get2byte(&data[pc+2]);
         1896  +      nFree = nFree + size;
         1897  +      if( next<=pc+size+3 ) break;
         1898  +      pc = next;
         1899  +    }
         1900  +    if( next>0 ){
         1901  +      /* Freeblock not in ascending order */
         1902  +      return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1903  +    }
         1904  +    if( pc+size>(unsigned int)usableSize ){
         1905  +      /* Last freeblock extends past page end */
         1906  +      return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1907  +    }
         1908  +  }
         1909  +
         1910  +  /* At this point, nFree contains the sum of the offset to the start
         1911  +  ** of the cell-content area plus the number of free bytes within
         1912  +  ** the cell-content area. If this is greater than the usable-size
         1913  +  ** of the page, then the page must be corrupted. This check also
         1914  +  ** serves to verify that the offset to the start of the cell-content
         1915  +  ** area, according to the page header, lies within the page.
         1916  +  */
         1917  +  if( nFree>usableSize ){
         1918  +    return SQLITE_CORRUPT_PGNO(pPage->pgno);
         1919  +  }
         1920  +  pPage->nFree = (u16)(nFree - iCellFirst);
         1921  +  pPage->isInit = 1;
  1918   1922     return SQLITE_OK;
  1919   1923   }
  1920   1924   
  1921   1925   /*
  1922   1926   ** Set up a raw page so that it looks like a database page holding
  1923   1927   ** no entries.
  1924   1928   */
................................................................................
  2074   2078     }
  2075   2079     assert( (*ppPage)->pgno==pgno );
  2076   2080     assert( (*ppPage)->aData==sqlite3PagerGetData(pDbPage) );
  2077   2081   
  2078   2082     /* If obtaining a child page for a cursor, we must verify that the page is
  2079   2083     ** compatible with the root page. */
  2080   2084     if( pCur && ((*ppPage)->nCell<1 || (*ppPage)->intKey!=pCur->curIntKey) ){
  2081         -    rc = SQLITE_CORRUPT_BKPT;
         2085  +    rc = SQLITE_CORRUPT_PGNO(pgno);
  2082   2086       releasePage(*ppPage);
  2083   2087       goto getAndInitPage_error;
  2084   2088     }
  2085   2089     return SQLITE_OK;
  2086   2090   
  2087   2091   getAndInitPage_error:
  2088   2092     if( pCur ) pCur->iPage--;
................................................................................
  3362   3366     int i;                             /* Counter variable */
  3363   3367     int nCell;                         /* Number of cells in page pPage */
  3364   3368     int rc;                            /* Return code */
  3365   3369     BtShared *pBt = pPage->pBt;
  3366   3370     Pgno pgno = pPage->pgno;
  3367   3371   
  3368   3372     assert( sqlite3_mutex_held(pPage->pBt->mutex) );
  3369         -  rc = btreeInitPage(pPage);
         3373  +  rc = pPage->isInit ? SQLITE_OK : btreeInitPage(pPage);
  3370   3374     if( rc!=SQLITE_OK ) return rc;
  3371   3375     nCell = pPage->nCell;
  3372   3376   
  3373   3377     for(i=0; i<nCell; i++){
  3374   3378       u8 *pCell = findCell(pPage, i);
  3375   3379   
  3376   3380       ptrmapPutOvflPtr(pPage, pCell, &rc);
................................................................................
  3405   3409   */
  3406   3410   static int modifyPagePointer(MemPage *pPage, Pgno iFrom, Pgno iTo, u8 eType){
  3407   3411     assert( sqlite3_mutex_held(pPage->pBt->mutex) );
  3408   3412     assert( sqlite3PagerIswriteable(pPage->pDbPage) );
  3409   3413     if( eType==PTRMAP_OVERFLOW2 ){
  3410   3414       /* The pointer is always the first 4 bytes of the page in this case.  */
  3411   3415       if( get4byte(pPage->aData)!=iFrom ){
  3412         -      return SQLITE_CORRUPT_BKPT;
         3416  +      return SQLITE_CORRUPT_PGNO(pPage->pgno);
  3413   3417       }
  3414   3418       put4byte(pPage->aData, iTo);
  3415   3419     }else{
  3416   3420       int i;
  3417   3421       int nCell;
  3418   3422       int rc;
  3419   3423   
  3420         -    rc = btreeInitPage(pPage);
         3424  +    rc = pPage->isInit ? SQLITE_OK : btreeInitPage(pPage);
  3421   3425       if( rc ) return rc;
  3422   3426       nCell = pPage->nCell;
  3423   3427   
  3424   3428       for(i=0; i<nCell; i++){
  3425   3429         u8 *pCell = findCell(pPage, i);
  3426   3430         if( eType==PTRMAP_OVERFLOW1 ){
  3427   3431           CellInfo info;
  3428   3432           pPage->xParseCell(pPage, pCell, &info);
  3429   3433           if( info.nLocal<info.nPayload ){
  3430   3434             if( pCell+info.nSize > pPage->aData+pPage->pBt->usableSize ){
  3431         -            return SQLITE_CORRUPT_BKPT;
         3435  +            return SQLITE_CORRUPT_PGNO(pPage->pgno);
  3432   3436             }
  3433   3437             if( iFrom==get4byte(pCell+info.nSize-4) ){
  3434   3438               put4byte(pCell+info.nSize-4, iTo);
  3435   3439               break;
  3436   3440             }
  3437   3441           }
  3438   3442         }else{
................................................................................
  3442   3446           }
  3443   3447         }
  3444   3448       }
  3445   3449     
  3446   3450       if( i==nCell ){
  3447   3451         if( eType!=PTRMAP_BTREE || 
  3448   3452             get4byte(&pPage->aData[pPage->hdrOffset+8])!=iFrom ){
  3449         -        return SQLITE_CORRUPT_BKPT;
         3453  +        return SQLITE_CORRUPT_PGNO(pPage->pgno);
  3450   3454         }
  3451   3455         put4byte(&pPage->aData[pPage->hdrOffset+8], iTo);
  3452   3456       }
  3453   3457     }
  3454   3458     return SQLITE_OK;
  3455   3459   }
  3456   3460   
................................................................................
  4550   4554     assert( aPayload > pPage->aData );
  4551   4555     if( (uptr)(aPayload - pPage->aData) > (pBt->usableSize - pCur->info.nLocal) ){
  4552   4556       /* Trying to read or write past the end of the data is an error.  The
  4553   4557       ** conditional above is really:
  4554   4558       **    &aPayload[pCur->info.nLocal] > &pPage->aData[pBt->usableSize]
  4555   4559       ** but is recast into its current form to avoid integer overflow problems
  4556   4560       */
  4557         -    return SQLITE_CORRUPT_BKPT;
         4561  +    return SQLITE_CORRUPT_PGNO(pPage->pgno);
  4558   4562     }
  4559   4563   
  4560   4564     /* Check if data must be read/written to/from the btree page itself. */
  4561   4565     if( offset<pCur->info.nLocal ){
  4562   4566       int a = amt;
  4563   4567       if( a+offset>pCur->info.nLocal ){
  4564   4568         a = pCur->info.nLocal - offset;
................................................................................
  4697   4701         }
  4698   4702         if( rc ) break;
  4699   4703         iIdx++;
  4700   4704       }
  4701   4705     }
  4702   4706   
  4703   4707     if( rc==SQLITE_OK && amt>0 ){
  4704         -    return SQLITE_CORRUPT_BKPT; /* Overflow chain ends prematurely */
         4708  +    /* Overflow chain ends prematurely */
         4709  +    return SQLITE_CORRUPT_PGNO(pPage->pgno);
  4705   4710     }
  4706   4711     return rc;
  4707   4712   }
  4708   4713   
  4709   4714   /*
  4710   4715   ** Read part of the payload for the row at which that cursor pCur is currently
  4711   4716   ** pointing.  "amt" bytes will be transferred into pBuf[].  The transfer
................................................................................
  4963   4968     ** Earlier versions of SQLite assumed that this test could not fail
  4964   4969     ** if the root page was already loaded when this function was called (i.e.
  4965   4970     ** if pCur->iPage>=0). But this is not so if the database is corrupted 
  4966   4971     ** in such a way that page pRoot is linked into a second b-tree table 
  4967   4972     ** (or the freelist).  */
  4968   4973     assert( pRoot->intKey==1 || pRoot->intKey==0 );
  4969   4974     if( pRoot->isInit==0 || (pCur->pKeyInfo==0)!=pRoot->intKey ){
  4970         -    return SQLITE_CORRUPT_BKPT;
         4975  +    return SQLITE_CORRUPT_PGNO(pCur->apPage[pCur->iPage]->pgno);
  4971   4976     }
  4972   4977   
  4973   4978   skip_init:  
  4974   4979     pCur->ix = 0;
  4975   4980     pCur->info.nSize = 0;
  4976   4981     pCur->curFlags &= ~(BTCF_AtLast|BTCF_ValidNKey|BTCF_ValidOvfl);
  4977   4982   
................................................................................
  5168   5173         if( (pCur->curFlags & BTCF_AtLast)!=0 ){
  5169   5174           *pRes = -1;
  5170   5175           return SQLITE_OK;
  5171   5176         }
  5172   5177         /* If the requested key is one more than the previous key, then
  5173   5178         ** try to get there using sqlite3BtreeNext() rather than a full
  5174   5179         ** binary search.  This is an optimization only.  The correct answer
  5175         -      ** is still obtained without this ase, only a little more slowely */
         5180  +      ** is still obtained without this case, only a little more slowely */
  5176   5181         if( pCur->info.nKey+1==intKey && !pCur->skipNext ){
  5177   5182           *pRes = 0;
  5178         -        rc = sqlite3BtreeNext(pCur, pRes);
  5179         -        if( rc ) return rc;
  5180         -        if( *pRes==0 ){
         5183  +        rc = sqlite3BtreeNext(pCur, 0);
         5184  +        if( rc==SQLITE_OK ){
  5181   5185             getCellInfo(pCur);
  5182   5186             if( pCur->info.nKey==intKey ){
  5183   5187               return SQLITE_OK;
  5184   5188             }
         5189  +        }else if( rc==SQLITE_DONE ){
         5190  +          rc = SQLITE_OK;
         5191  +        }else{
         5192  +          return rc;
  5185   5193           }
  5186   5194         }
  5187   5195       }
  5188   5196     }
  5189   5197   
  5190   5198     if( pIdxKey ){
  5191   5199       xRecordCompare = sqlite3VdbeFindCompare(pIdxKey);
................................................................................
  5233   5241       pCur->ix = (u16)idx;
  5234   5242       if( xRecordCompare==0 ){
  5235   5243         for(;;){
  5236   5244           i64 nCellKey;
  5237   5245           pCell = findCellPastPtr(pPage, idx);
  5238   5246           if( pPage->intKeyLeaf ){
  5239   5247             while( 0x80 <= *(pCell++) ){
  5240         -            if( pCell>=pPage->aDataEnd ) return SQLITE_CORRUPT_BKPT;
         5248  +            if( pCell>=pPage->aDataEnd ){
         5249  +              return SQLITE_CORRUPT_PGNO(pPage->pgno);
         5250  +            }
  5241   5251             }
  5242   5252           }
  5243   5253           getVarint(pCell, (u64*)&nCellKey);
  5244   5254           if( nCellKey<intKey ){
  5245   5255             lwr = idx+1;
  5246   5256             if( lwr>upr ){ c = -1; break; }
  5247   5257           }else if( nCellKey>intKey ){
................................................................................
  5306   5316             pPage->xParseCell(pPage, pCellBody, &pCur->info);
  5307   5317             nCell = (int)pCur->info.nKey;
  5308   5318             testcase( nCell<0 );   /* True if key size is 2^32 or more */
  5309   5319             testcase( nCell==0 );  /* Invalid key size:  0x80 0x80 0x00 */
  5310   5320             testcase( nCell==1 );  /* Invalid key size:  0x80 0x80 0x01 */
  5311   5321             testcase( nCell==2 );  /* Minimum legal index key size */
  5312   5322             if( nCell<2 ){
  5313         -            rc = SQLITE_CORRUPT_BKPT;
         5323  +            rc = SQLITE_CORRUPT_PGNO(pPage->pgno);
  5314   5324               goto moveto_finish;
  5315   5325             }
  5316   5326             pCellKey = sqlite3Malloc( nCell+18 );
  5317   5327             if( pCellKey==0 ){
  5318   5328               rc = SQLITE_NOMEM_BKPT;
  5319   5329               goto moveto_finish;
  5320   5330             }
................................................................................
  5411   5421     for(n=1, i=0; i<=pCur->iPage; i++){
  5412   5422       n *= pCur->apPage[i]->nCell;
  5413   5423     }
  5414   5424     return n;
  5415   5425   }
  5416   5426   
  5417   5427   /*
  5418         -** Advance the cursor to the next entry in the database.  If
  5419         -** successful then set *pRes=0.  If the cursor
  5420         -** was already pointing to the last entry in the database before
  5421         -** this routine was called, then set *pRes=1.
         5428  +** Advance the cursor to the next entry in the database. 
         5429  +** Return value:
         5430  +**
         5431  +**    SQLITE_OK        success
         5432  +**    SQLITE_DONE      cursor is already pointing at the last element
         5433  +**    otherwise        some kind of error occurred
  5422   5434   **
  5423   5435   ** The main entry point is sqlite3BtreeNext().  That routine is optimized
  5424   5436   ** for the common case of merely incrementing the cell counter BtCursor.aiIdx
  5425   5437   ** to the next cell on the current page.  The (slower) btreeNext() helper
  5426   5438   ** routine is called when it is necessary to move to a different page or
  5427   5439   ** to restore the cursor.
  5428   5440   **
  5429         -** The calling function will set *pRes to 0 or 1.  The initial *pRes value
  5430         -** will be 1 if the cursor being stepped corresponds to an SQL index and
  5431         -** if this routine could have been skipped if that SQL index had been
  5432         -** a unique index.  Otherwise the caller will have set *pRes to zero.
  5433         -** Zero is the common case. The btree implementation is free to use the
  5434         -** initial *pRes value as a hint to improve performance, but the current
  5435         -** SQLite btree implementation does not. (Note that the comdb2 btree
  5436         -** implementation does use this hint, however.)
         5441  +** If bit 0x01 of the flags argument is 1, then the cursor corresponds to
         5442  +** an SQL index and this routine could have been skipped if the SQL index
         5443  +** had been a unique index.  The flags argument is a hint to the implement.
         5444  +** SQLite btree implementation does not use this hint, but COMDB2 does.
  5437   5445   */
  5438         -static SQLITE_NOINLINE int btreeNext(BtCursor *pCur, int *pRes){
         5446  +static SQLITE_NOINLINE int btreeNext(BtCursor *pCur, int flags){
  5439   5447     int rc;
  5440   5448     int idx;
  5441   5449     MemPage *pPage;
  5442   5450   
  5443   5451     assert( cursorOwnsBtShared(pCur) );
  5444   5452     assert( pCur->skipNext==0 || pCur->eState!=CURSOR_VALID );
  5445         -  assert( *pRes==0 );
         5453  +  assert( flags==0 );
  5446   5454     if( pCur->eState!=CURSOR_VALID ){
  5447   5455       assert( (pCur->curFlags & BTCF_ValidOvfl)==0 );
  5448   5456       rc = restoreCursorPosition(pCur);
  5449   5457       if( rc!=SQLITE_OK ){
  5450   5458         return rc;
  5451   5459       }
  5452   5460       if( CURSOR_INVALID==pCur->eState ){
  5453         -      *pRes = 1;
  5454         -      return SQLITE_OK;
         5461  +      return SQLITE_DONE;
  5455   5462       }
  5456   5463       if( pCur->skipNext ){
  5457   5464         assert( pCur->eState==CURSOR_VALID || pCur->eState==CURSOR_SKIPNEXT );
  5458   5465         pCur->eState = CURSOR_VALID;
  5459   5466         if( pCur->skipNext>0 ){
  5460   5467           pCur->skipNext = 0;
  5461   5468           return SQLITE_OK;
................................................................................
  5479   5486       if( !pPage->leaf ){
  5480   5487         rc = moveToChild(pCur, get4byte(&pPage->aData[pPage->hdrOffset+8]));
  5481   5488         if( rc ) return rc;
  5482   5489         return moveToLeftmost(pCur);
  5483   5490       }
  5484   5491       do{
  5485   5492         if( pCur->iPage==0 ){
  5486         -        *pRes = 1;
  5487   5493           pCur->eState = CURSOR_INVALID;
  5488         -        return SQLITE_OK;
         5494  +        return SQLITE_DONE;
  5489   5495         }
  5490   5496         moveToParent(pCur);
  5491   5497         pPage = pCur->apPage[pCur->iPage];
  5492   5498       }while( pCur->ix>=pPage->nCell );
  5493   5499       if( pPage->intKey ){
  5494         -      return sqlite3BtreeNext(pCur, pRes);
         5500  +      return sqlite3BtreeNext(pCur, flags);
  5495   5501       }else{
  5496   5502         return SQLITE_OK;
  5497   5503       }
  5498   5504     }
  5499   5505     if( pPage->leaf ){
  5500   5506       return SQLITE_OK;
  5501   5507     }else{
  5502   5508       return moveToLeftmost(pCur);
  5503   5509     }
  5504   5510   }
  5505         -int sqlite3BtreeNext(BtCursor *pCur, int *pRes){
         5511  +int sqlite3BtreeNext(BtCursor *pCur, int flags){
  5506   5512     MemPage *pPage;
  5507   5513     assert( cursorOwnsBtShared(pCur) );
  5508         -  assert( pRes!=0 );
  5509         -  assert( *pRes==0 || *pRes==1 );
         5514  +  assert( flags==0 || flags==1 );
  5510   5515     assert( pCur->skipNext==0 || pCur->eState!=CURSOR_VALID );
  5511   5516     pCur->info.nSize = 0;
  5512   5517     pCur->curFlags &= ~(BTCF_ValidNKey|BTCF_ValidOvfl);
  5513         -  *pRes = 0;
  5514         -  if( pCur->eState!=CURSOR_VALID ) return btreeNext(pCur, pRes);
         5518  +  if( pCur->eState!=CURSOR_VALID ) return btreeNext(pCur, 0);
  5515   5519     pPage = pCur->apPage[pCur->iPage];
  5516   5520     if( (++pCur->ix)>=pPage->nCell ){
  5517   5521       pCur->ix--;
  5518         -    return btreeNext(pCur, pRes);
         5522  +    return btreeNext(pCur, 0);
  5519   5523     }
  5520   5524     if( pPage->leaf ){
  5521   5525       return SQLITE_OK;
  5522   5526     }else{
  5523   5527       return moveToLeftmost(pCur);
  5524   5528     }
  5525   5529   }
  5526   5530   
  5527   5531   /*
  5528         -** Step the cursor to the back to the previous entry in the database.  If
  5529         -** successful then set *pRes=0.  If the cursor
  5530         -** was already pointing to the first entry in the database before
  5531         -** this routine was called, then set *pRes=1.
         5532  +** Step the cursor to the back to the previous entry in the database.
         5533  +** Return values:
         5534  +**
         5535  +**     SQLITE_OK     success
         5536  +**     SQLITE_DONE   the cursor is already on the first element of the table
         5537  +**     otherwise     some kind of error occurred
  5532   5538   **
  5533   5539   ** The main entry point is sqlite3BtreePrevious().  That routine is optimized
  5534   5540   ** for the common case of merely decrementing the cell counter BtCursor.aiIdx
  5535   5541   ** to the previous cell on the current page.  The (slower) btreePrevious()
  5536   5542   ** helper routine is called when it is necessary to move to a different page
  5537   5543   ** or to restore the cursor.
  5538   5544   **
  5539         -** The calling function will set *pRes to 0 or 1.  The initial *pRes value
  5540         -** will be 1 if the cursor being stepped corresponds to an SQL index and
  5541         -** if this routine could have been skipped if that SQL index had been
  5542         -** a unique index.  Otherwise the caller will have set *pRes to zero.
  5543         -** Zero is the common case. The btree implementation is free to use the
  5544         -** initial *pRes value as a hint to improve performance, but the current
  5545         -** SQLite btree implementation does not. (Note that the comdb2 btree
  5546         -** implementation does use this hint, however.)
         5545  +**
         5546  +** If bit 0x01 of the flags argument is 1, then the cursor corresponds to
         5547  +** an SQL index and this routine could have been skipped if the SQL index
         5548  +** had been a unique index.  The flags argument is a hint to the implement.
         5549  +** SQLite btree implementation does not use this hint, but COMDB2 does.
  5547   5550   */
  5548         -static SQLITE_NOINLINE int btreePrevious(BtCursor *pCur, int *pRes){
         5551  +static SQLITE_NOINLINE int btreePrevious(BtCursor *pCur, int flags){
  5549   5552     int rc;
  5550   5553     MemPage *pPage;
  5551   5554   
  5552   5555     assert( cursorOwnsBtShared(pCur) );
  5553         -  assert( pRes!=0 );
  5554         -  assert( *pRes==0 );
         5556  +  assert( flags==0 );
  5555   5557     assert( pCur->skipNext==0 || pCur->eState!=CURSOR_VALID );
  5556   5558     assert( (pCur->curFlags & (BTCF_AtLast|BTCF_ValidOvfl|BTCF_ValidNKey))==0 );
  5557   5559     assert( pCur->info.nSize==0 );
  5558   5560     if( pCur->eState!=CURSOR_VALID ){
  5559   5561       rc = restoreCursorPosition(pCur);
  5560   5562       if( rc!=SQLITE_OK ){
  5561   5563         return rc;
  5562   5564       }
  5563   5565       if( CURSOR_INVALID==pCur->eState ){
  5564         -      *pRes = 1;
  5565         -      return SQLITE_OK;
         5566  +      return SQLITE_DONE;
  5566   5567       }
  5567   5568       if( pCur->skipNext ){
  5568   5569         assert( pCur->eState==CURSOR_VALID || pCur->eState==CURSOR_SKIPNEXT );
  5569   5570         pCur->eState = CURSOR_VALID;
  5570   5571         if( pCur->skipNext<0 ){
  5571   5572           pCur->skipNext = 0;
  5572   5573           return SQLITE_OK;
................................................................................
  5582   5583       rc = moveToChild(pCur, get4byte(findCell(pPage, idx)));
  5583   5584       if( rc ) return rc;
  5584   5585       rc = moveToRightmost(pCur);
  5585   5586     }else{
  5586   5587       while( pCur->ix==0 ){
  5587   5588         if( pCur->iPage==0 ){
  5588   5589           pCur->eState = CURSOR_INVALID;
  5589         -        *pRes = 1;
  5590         -        return SQLITE_OK;
         5590  +        return SQLITE_DONE;
  5591   5591         }
  5592   5592         moveToParent(pCur);
  5593   5593       }
  5594   5594       assert( pCur->info.nSize==0 );
  5595   5595       assert( (pCur->curFlags & (BTCF_ValidOvfl))==0 );
  5596   5596   
  5597   5597       pCur->ix--;
  5598   5598       pPage = pCur->apPage[pCur->iPage];
  5599   5599       if( pPage->intKey && !pPage->leaf ){
  5600         -      rc = sqlite3BtreePrevious(pCur, pRes);
         5600  +      rc = sqlite3BtreePrevious(pCur, flags);
  5601   5601       }else{
  5602   5602         rc = SQLITE_OK;
  5603   5603       }
  5604   5604     }
  5605   5605     return rc;
  5606   5606   }
  5607         -int sqlite3BtreePrevious(BtCursor *pCur, int *pRes){
         5607  +int sqlite3BtreePrevious(BtCursor *pCur, int flags){
  5608   5608     assert( cursorOwnsBtShared(pCur) );
  5609         -  assert( pRes!=0 );
  5610         -  assert( *pRes==0 || *pRes==1 );
         5609  +  assert( flags==0 || flags==1 );
  5611   5610     assert( pCur->skipNext==0 || pCur->eState!=CURSOR_VALID );
  5612         -  *pRes = 0;
  5613   5611     pCur->curFlags &= ~(BTCF_AtLast|BTCF_ValidOvfl|BTCF_ValidNKey);
  5614   5612     pCur->info.nSize = 0;
  5615   5613     if( pCur->eState!=CURSOR_VALID
  5616   5614      || pCur->ix==0
  5617   5615      || pCur->apPage[pCur->iPage]->leaf==0
  5618   5616     ){
  5619         -    return btreePrevious(pCur, pRes);
         5617  +    return btreePrevious(pCur, 0);
  5620   5618     }
  5621   5619     pCur->ix--;
  5622   5620     return SQLITE_OK;
  5623   5621   }
  5624   5622   
  5625   5623   /*
  5626   5624   ** Allocate a new page from the database file.
................................................................................
  5720   5718           /* EVIDENCE-OF: R-59841-13798 The 4-byte big-endian integer at offset 32
  5721   5719           ** stores the page number of the first page of the freelist, or zero if
  5722   5720           ** the freelist is empty. */
  5723   5721           iTrunk = get4byte(&pPage1->aData[32]);
  5724   5722         }
  5725   5723         testcase( iTrunk==mxPage );
  5726   5724         if( iTrunk>mxPage || nSearch++ > n ){
  5727         -        rc = SQLITE_CORRUPT_BKPT;
         5725  +        rc = SQLITE_CORRUPT_PGNO(pPrevTrunk ? pPrevTrunk->pgno : 1);
  5728   5726         }else{
  5729   5727           rc = btreeGetUnusedPage(pBt, iTrunk, &pTrunk, 0);
  5730   5728         }
  5731   5729         if( rc ){
  5732   5730           pTrunk = 0;
  5733   5731           goto end_allocate_page;
  5734   5732         }
................................................................................
  5749   5747           *pPgno = iTrunk;
  5750   5748           memcpy(&pPage1->aData[32], &pTrunk->aData[0], 4);
  5751   5749           *ppPage = pTrunk;
  5752   5750           pTrunk = 0;
  5753   5751           TRACE(("ALLOCATE: %d trunk - %d free pages left\n", *pPgno, n-1));
  5754   5752         }else if( k>(u32)(pBt->usableSize/4 - 2) ){
  5755   5753           /* Value of k is out of range.  Database corruption */
  5756         -        rc = SQLITE_CORRUPT_BKPT;
         5754  +        rc = SQLITE_CORRUPT_PGNO(iTrunk);
  5757   5755           goto end_allocate_page;
  5758   5756   #ifndef SQLITE_OMIT_AUTOVACUUM
  5759   5757         }else if( searchList 
  5760   5758               && (nearby==iTrunk || (iTrunk<nearby && eMode==BTALLOC_LE)) 
  5761   5759         ){
  5762   5760           /* The list is being searched and this trunk page is the page
  5763   5761           ** to allocate, regardless of whether it has leaves.
................................................................................
  5783   5781             /* The trunk page is required by the caller but it contains 
  5784   5782             ** pointers to free-list leaves. The first leaf becomes a trunk
  5785   5783             ** page in this case.
  5786   5784             */
  5787   5785             MemPage *pNewTrunk;
  5788   5786             Pgno iNewTrunk = get4byte(&pTrunk->aData[8]);
  5789   5787             if( iNewTrunk>mxPage ){ 
  5790         -            rc = SQLITE_CORRUPT_BKPT;
         5788  +            rc = SQLITE_CORRUPT_PGNO(iTrunk);
  5791   5789               goto end_allocate_page;
  5792   5790             }
  5793   5791             testcase( iNewTrunk==mxPage );
  5794   5792             rc = btreeGetUnusedPage(pBt, iNewTrunk, &pNewTrunk, 0);
  5795   5793             if( rc!=SQLITE_OK ){
  5796   5794               goto end_allocate_page;
  5797   5795             }
................................................................................
  5848   5846           }else{
  5849   5847             closest = 0;
  5850   5848           }
  5851   5849   
  5852   5850           iPage = get4byte(&aData[8+closest*4]);
  5853   5851           testcase( iPage==mxPage );
  5854   5852           if( iPage>mxPage ){
  5855         -          rc = SQLITE_CORRUPT_BKPT;
         5853  +          rc = SQLITE_CORRUPT_PGNO(iTrunk);
  5856   5854             goto end_allocate_page;
  5857   5855           }
  5858   5856           testcase( iPage==mxPage );
  5859   5857           if( !searchList 
  5860   5858            || (iPage==nearby || (iPage<nearby && eMode==BTALLOC_LE)) 
  5861   5859           ){
  5862   5860             int noContent;
................................................................................
  6118   6116   
  6119   6117     assert( sqlite3_mutex_held(pPage->pBt->mutex) );
  6120   6118     pPage->xParseCell(pPage, pCell, pInfo);
  6121   6119     if( pInfo->nLocal==pInfo->nPayload ){
  6122   6120       return SQLITE_OK;  /* No overflow pages. Return without doing anything */
  6123   6121     }
  6124   6122     if( pCell+pInfo->nSize-1 > pPage->aData+pPage->maskPage ){
  6125         -    return SQLITE_CORRUPT_BKPT;  /* Cell extends past end of page */
         6123  +    /* Cell extends past end of page */
         6124  +    return SQLITE_CORRUPT_PGNO(pPage->pgno);
  6126   6125     }
  6127   6126     ovflPgno = get4byte(pCell + pInfo->nSize - 4);
  6128   6127     assert( pBt->usableSize > 4 );
  6129   6128     ovflPageSize = pBt->usableSize - 4;
  6130   6129     nOvfl = (pInfo->nPayload - pInfo->nLocal + ovflPageSize - 1)/ovflPageSize;
  6131   6130     assert( nOvfl>0 || 
  6132   6131       (CORRUPT_DB && (pInfo->nPayload + ovflPageSize)<ovflPageSize)
................................................................................
  8333   8332     ** the cursor to the largest entry in the tree that is smaller than
  8334   8333     ** the entry being deleted. This cell will replace the cell being deleted
  8335   8334     ** from the internal node. The 'previous' entry is used for this instead
  8336   8335     ** of the 'next' entry, as the previous entry is always a part of the
  8337   8336     ** sub-tree headed by the child page of the cell being deleted. This makes
  8338   8337     ** balancing the tree following the delete operation easier.  */
  8339   8338     if( !pPage->leaf ){
  8340         -    int notUsed = 0;
  8341         -    rc = sqlite3BtreePrevious(pCur, &notUsed);
         8339  +    rc = sqlite3BtreePrevious(pCur, 0);
         8340  +    assert( rc!=SQLITE_DONE );
  8342   8341       if( rc ) return rc;
  8343   8342     }
  8344   8343   
  8345   8344     /* Save the positions of any other cursors open on this table before
  8346   8345     ** making any modifications.  */
  8347   8346     if( pCur->curFlags & BTCF_Multiple ){
  8348   8347       rc = saveAllCursors(pBt, pCur->pgnoRoot, pCur);

Changes to src/btree.h.

   282    282     int nZero;              /* Extra zero data appended after pData,nData */
   283    283   };
   284    284   
   285    285   int sqlite3BtreeInsert(BtCursor*, const BtreePayload *pPayload,
   286    286                          int flags, int seekResult);
   287    287   int sqlite3BtreeFirst(BtCursor*, int *pRes);
   288    288   int sqlite3BtreeLast(BtCursor*, int *pRes);
   289         -int sqlite3BtreeNext(BtCursor*, int *pRes);
          289  +int sqlite3BtreeNext(BtCursor*, int flags);
   290    290   int sqlite3BtreeEof(BtCursor*);
   291         -int sqlite3BtreePrevious(BtCursor*, int *pRes);
          291  +int sqlite3BtreePrevious(BtCursor*, int flags);
   292    292   i64 sqlite3BtreeIntegerKey(BtCursor*);
   293    293   int sqlite3BtreePayload(BtCursor*, u32 offset, u32 amt, void*);
   294    294   const void *sqlite3BtreePayloadFetch(BtCursor*, u32 *pAmt);
   295    295   u32 sqlite3BtreePayloadSize(BtCursor*);
   296    296   
   297    297   char *sqlite3BtreeIntegrityCheck(Btree*, int *aRoot, int nRoot, int, int*);
   298    298   struct Pager *sqlite3BtreePager(Btree*);

Changes to src/build.c.

   935    935       pParse->nErr++;
   936    936       goto begin_table_error;
   937    937     }
   938    938     pTable->zName = zName;
   939    939     pTable->iPKey = -1;
   940    940     pTable->pSchema = db->aDb[iDb].pSchema;
   941    941     pTable->nTabRef = 1;
          942  +#ifdef SQLITE_DEFAULT_ROWEST
          943  +  pTable->nRowLogEst = sqlite3LogEst(SQLITE_DEFAULT_ROWEST);
          944  +#else
   942    945     pTable->nRowLogEst = 200; assert( 200==sqlite3LogEst(1048576) );
          946  +#endif
   943    947     assert( pParse->pNewTable==0 );
   944    948     pParse->pNewTable = pTable;
   945    949   
   946    950     /* If this is the magic sqlite_sequence table used by autoincrement,
   947    951     ** then record a pointer to this table in the main database structure
   948    952     ** so that INSERT can find the table easily.
   949    953     */

Changes to src/ctime.c.

    12     12   **
    13     13   ** This file implements routines used to report what compile-time options
    14     14   ** SQLite was built with.
    15     15   */
    16     16   
    17     17   #ifndef SQLITE_OMIT_COMPILEOPTION_DIAGS
    18     18   
    19         -#include "sqliteInt.h"
           19  +
           20  +/* These macros are provided to "stringify" the value of the define
           21  +** for those options in which the value is meaningful. */
           22  +#define CTIMEOPT_VAL_(opt) #opt
           23  +#define CTIMEOPT_VAL(opt) CTIMEOPT_VAL_(opt)
    20     24   
    21     25   /*
    22     26   ** An array of names of all compile-time options.  This array should 
    23     27   ** be sorted A-Z.
    24     28   **
    25     29   ** This array looks large, but in a typical installation actually uses
    26     30   ** only a handful of compile-time options, so most times this array is usually
    27     31   ** rather short and uses little memory space.
    28     32   */
    29         -static const char * const azCompileOpt[] = {
           33  +static const char * const sqlite3azCompileOpt[] = {
    30     34   
    31         -/* These macros are provided to "stringify" the value of the define
    32         -** for those options in which the value is meaningful. */
    33         -#define CTIMEOPT_VAL_(opt) #opt
    34         -#define CTIMEOPT_VAL(opt) CTIMEOPT_VAL_(opt)
    35         -
           35  +/* 
           36  +** BEGIN CODE GENERATED BY tool/mkctime.tcl 
           37  +*/
    36     38   #if SQLITE_32BIT_ROWID
    37     39     "32BIT_ROWID",
    38     40   #endif
    39     41   #if SQLITE_4_BYTE_ALIGNED_MALLOC
    40     42     "4_BYTE_ALIGNED_MALLOC",
    41     43   #endif
           44  +#if SQLITE_64BIT_STATS
           45  +  "64BIT_STATS",
           46  +#endif
           47  +#if SQLITE_ALLOW_COVERING_INDEX_SCAN
           48  +  "ALLOW_COVERING_INDEX_SCAN",
           49  +#endif
           50  +#if SQLITE_ALLOW_URI_AUTHORITY
           51  +  "ALLOW_URI_AUTHORITY",
           52  +#endif
           53  +#ifdef SQLITE_BITMASK_TYPE
           54  +  "BITMASK_TYPE=" CTIMEOPT_VAL(SQLITE_BITMASK_TYPE),
           55  +#endif
           56  +#if SQLITE_BUG_COMPATIBLE_20160819
           57  +  "BUG_COMPATIBLE_20160819",
           58  +#endif
    42     59   #if SQLITE_CASE_SENSITIVE_LIKE
    43     60     "CASE_SENSITIVE_LIKE",
    44     61   #endif
    45     62   #if SQLITE_CHECK_PAGES
    46     63     "CHECK_PAGES",
    47     64   #endif
    48     65   #if defined(__clang__) && defined(__clang_major__)
................................................................................
    53     70     "COMPILER=msvc-" CTIMEOPT_VAL(_MSC_VER),
    54     71   #elif defined(__GNUC__) && defined(__VERSION__)
    55     72     "COMPILER=gcc-" __VERSION__,
    56     73   #endif
    57     74   #if SQLITE_COVERAGE_TEST
    58     75     "COVERAGE_TEST",
    59     76   #endif
    60         -#ifdef SQLITE_DEBUG
           77  +#if SQLITE_DEBUG
    61     78     "DEBUG",
    62     79   #endif
    63         -#if SQLITE_DEFAULT_LOCKING_MODE
           80  +#if SQLITE_DEFAULT_AUTOMATIC_INDEX
           81  +  "DEFAULT_AUTOMATIC_INDEX",
           82  +#endif
           83  +#if SQLITE_DEFAULT_AUTOVACUUM
           84  +  "DEFAULT_AUTOVACUUM",
           85  +#endif
           86  +#ifdef SQLITE_DEFAULT_CACHE_SIZE
           87  +  "DEFAULT_CACHE_SIZE=" CTIMEOPT_VAL(SQLITE_DEFAULT_CACHE_SIZE),
           88  +#endif
           89  +#if SQLITE_DEFAULT_CKPTFULLFSYNC
           90  +  "DEFAULT_CKPTFULLFSYNC",
           91  +#endif
           92  +#ifdef SQLITE_DEFAULT_FILE_FORMAT
           93  +  "DEFAULT_FILE_FORMAT=" CTIMEOPT_VAL(SQLITE_DEFAULT_FILE_FORMAT),
           94  +#endif
           95  +#ifdef SQLITE_DEFAULT_FILE_PERMISSIONS
           96  +  "DEFAULT_FILE_PERMISSIONS=" CTIMEOPT_VAL(SQLITE_DEFAULT_FILE_PERMISSIONS),
           97  +#endif
           98  +#if SQLITE_DEFAULT_FOREIGN_KEYS
           99  +  "DEFAULT_FOREIGN_KEYS",
          100  +#endif
          101  +#ifdef SQLITE_DEFAULT_JOURNAL_SIZE_LIMIT
          102  +  "DEFAULT_JOURNAL_SIZE_LIMIT=" CTIMEOPT_VAL(SQLITE_DEFAULT_JOURNAL_SIZE_LIMIT),
          103  +#endif
          104  +#ifdef SQLITE_DEFAULT_LOCKING_MODE
    64    105     "DEFAULT_LOCKING_MODE=" CTIMEOPT_VAL(SQLITE_DEFAULT_LOCKING_MODE),
    65    106   #endif
    66         -#if defined(SQLITE_DEFAULT_MMAP_SIZE) && !defined(SQLITE_DEFAULT_MMAP_SIZE_xc)
          107  +#ifdef SQLITE_DEFAULT_LOOKASIDE
          108  +  "DEFAULT_LOOKASIDE=" CTIMEOPT_VAL(SQLITE_DEFAULT_LOOKASIDE),
          109  +#endif
          110  +#if SQLITE_DEFAULT_MEMSTATUS
          111  +  "DEFAULT_MEMSTATUS",
          112  +#endif
          113  +#ifdef SQLITE_DEFAULT_MMAP_SIZE
    67    114     "DEFAULT_MMAP_SIZE=" CTIMEOPT_VAL(SQLITE_DEFAULT_MMAP_SIZE),
    68    115   #endif
    69         -#if SQLITE_DEFAULT_SYNCHRONOUS
          116  +#ifdef SQLITE_DEFAULT_PAGE_SIZE
          117  +  "DEFAULT_PAGE_SIZE=" CTIMEOPT_VAL(SQLITE_DEFAULT_PAGE_SIZE),
          118  +#endif
          119  +#ifdef SQLITE_DEFAULT_PCACHE_INITSZ
          120  +  "DEFAULT_PCACHE_INITSZ=" CTIMEOPT_VAL(SQLITE_DEFAULT_PCACHE_INITSZ),
          121  +#endif
          122  +#ifdef SQLITE_DEFAULT_PROXYDIR_PERMISSIONS
          123  +  "DEFAULT_PROXYDIR_PERMISSIONS=" CTIMEOPT_VAL(SQLITE_DEFAULT_PROXYDIR_PERMISSIONS),
          124  +#endif
          125  +#if SQLITE_DEFAULT_RECURSIVE_TRIGGERS
          126  +  "DEFAULT_RECURSIVE_TRIGGERS",
          127  +#endif
          128  +#ifdef SQLITE_DEFAULT_ROWEST
          129  +  "DEFAULT_ROWEST=" CTIMEOPT_VAL(SQLITE_DEFAULT_ROWEST),
          130  +#endif
          131  +#ifdef SQLITE_DEFAULT_SECTOR_SIZE
          132  +  "DEFAULT_SECTOR_SIZE=" CTIMEOPT_VAL(SQLITE_DEFAULT_SECTOR_SIZE),
          133  +#endif
          134  +#ifdef SQLITE_DEFAULT_SYNCHRONOUS
    70    135     "DEFAULT_SYNCHRONOUS=" CTIMEOPT_VAL(SQLITE_DEFAULT_SYNCHRONOUS),
    71    136   #endif
    72         -#if SQLITE_DEFAULT_WAL_SYNCHRONOUS
          137  +#ifdef SQLITE_DEFAULT_WAL_AUTOCHECKPOINT
          138  +  "DEFAULT_WAL_AUTOCHECKPOINT=" CTIMEOPT_VAL(SQLITE_DEFAULT_WAL_AUTOCHECKPOINT),
          139  +#endif
          140  +#ifdef SQLITE_DEFAULT_WAL_SYNCHRONOUS
    73    141     "DEFAULT_WAL_SYNCHRONOUS=" CTIMEOPT_VAL(SQLITE_DEFAULT_WAL_SYNCHRONOUS),
    74    142   #endif
          143  +#ifdef SQLITE_DEFAULT_WORKER_THREADS
          144  +  "DEFAULT_WORKER_THREADS=" CTIMEOPT_VAL(SQLITE_DEFAULT_WORKER_THREADS),
          145  +#endif
    75    146   #if SQLITE_DIRECT_OVERFLOW_READ
    76    147     "DIRECT_OVERFLOW_READ",
    77    148   #endif
    78    149   #if SQLITE_DISABLE_DIRSYNC
    79    150     "DISABLE_DIRSYNC",
    80    151   #endif
          152  +#if SQLITE_DISABLE_FTS3_UNICODE
          153  +  "DISABLE_FTS3_UNICODE",
          154  +#endif
          155  +#if SQLITE_DISABLE_FTS4_DEFERRED
          156  +  "DISABLE_FTS4_DEFERRED",
          157  +#endif
          158  +#if SQLITE_DISABLE_INTRINSIC
          159  +  "DISABLE_INTRINSIC",
          160  +#endif
    81    161   #if SQLITE_DISABLE_LFS
    82    162     "DISABLE_LFS",
    83    163   #endif
    84         -#if SQLITE_ENABLE_8_3_NAMES
          164  +#if SQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS
          165  +  "DISABLE_PAGECACHE_OVERFLOW_STATS",
          166  +#endif
          167  +#if SQLITE_DISABLE_SKIPAHEAD_DISTINCT
          168  +  "DISABLE_SKIPAHEAD_DISTINCT",
          169  +#endif
          170  +#ifdef SQLITE_ENABLE_8_3_NAMES
    85    171     "ENABLE_8_3_NAMES=" CTIMEOPT_VAL(SQLITE_ENABLE_8_3_NAMES),
    86    172   #endif
    87    173   #if SQLITE_ENABLE_API_ARMOR
    88    174     "ENABLE_API_ARMOR",
    89    175   #endif
    90    176   #if SQLITE_ENABLE_ATOMIC_WRITE
    91    177     "ENABLE_ATOMIC_WRITE",
    92    178   #endif
    93    179   #if SQLITE_ENABLE_CEROD
    94    180     "ENABLE_CEROD",
    95    181   #endif
    96    182   #if SQLITE_ENABLE_COLUMN_METADATA
    97    183     "ENABLE_COLUMN_METADATA",
          184  +#endif
          185  +#if SQLITE_ENABLE_COLUMN_USED_MASK
          186  +  "ENABLE_COLUMN_USED_MASK",
          187  +#endif
          188  +#if SQLITE_ENABLE_COSTMULT
          189  +  "ENABLE_COSTMULT",
          190  +#endif
          191  +#if SQLITE_ENABLE_CURSOR_HINTS
          192  +  "ENABLE_CURSOR_HINTS",
    98    193   #endif
    99    194   #if SQLITE_ENABLE_DBSTAT_VTAB
   100    195     "ENABLE_DBSTAT_VTAB",
   101    196   #endif
   102    197   #if SQLITE_ENABLE_EXPENSIVE_ASSERT
   103    198     "ENABLE_EXPENSIVE_ASSERT",
   104    199   #endif
................................................................................
   110    205   #endif
   111    206   #if SQLITE_ENABLE_FTS3
   112    207     "ENABLE_FTS3",
   113    208   #endif
   114    209   #if SQLITE_ENABLE_FTS3_PARENTHESIS
   115    210     "ENABLE_FTS3_PARENTHESIS",
   116    211   #endif
          212  +#if SQLITE_ENABLE_FTS3_TOKENIZER
          213  +  "ENABLE_FTS3_TOKENIZER",
          214  +#endif
   117    215   #if SQLITE_ENABLE_FTS4
   118    216     "ENABLE_FTS4",
   119    217   #endif
   120    218   #if SQLITE_ENABLE_FTS5
   121    219     "ENABLE_FTS5",
          220  +#endif
          221  +#if SQLITE_ENABLE_HIDDEN_COLUMNS
          222  +  "ENABLE_HIDDEN_COLUMNS",
   122    223   #endif
   123    224   #if SQLITE_ENABLE_ICU
   124    225     "ENABLE_ICU",
   125    226   #endif
   126    227   #if SQLITE_ENABLE_IOTRACE
   127    228     "ENABLE_IOTRACE",
   128    229   #endif
   129    230   #if SQLITE_ENABLE_JSON1
   130    231     "ENABLE_JSON1",
   131    232   #endif
   132    233   #if SQLITE_ENABLE_LOAD_EXTENSION
   133    234     "ENABLE_LOAD_EXTENSION",
   134    235   #endif
   135         -#if SQLITE_ENABLE_LOCKING_STYLE
          236  +#ifdef SQLITE_ENABLE_LOCKING_STYLE
   136    237     "ENABLE_LOCKING_STYLE=" CTIMEOPT_VAL(SQLITE_ENABLE_LOCKING_STYLE),
   137    238   #endif
   138    239   #if SQLITE_ENABLE_MEMORY_MANAGEMENT
   139    240     "ENABLE_MEMORY_MANAGEMENT",
   140    241   #endif
   141    242   #if SQLITE_ENABLE_MEMSYS3
   142    243     "ENABLE_MEMSYS3",
   143    244   #endif
   144    245   #if SQLITE_ENABLE_MEMSYS5
   145    246     "ENABLE_MEMSYS5",
          247  +#endif
          248  +#if SQLITE_ENABLE_MULTIPLEX
          249  +  "ENABLE_MULTIPLEX",
          250  +#endif
          251  +#if SQLITE_ENABLE_NULL_TRIM
          252  +  "ENABLE_NULL_TRIM",
   146    253   #endif
   147    254   #if SQLITE_ENABLE_OVERSIZE_CELL_CHECK
   148    255     "ENABLE_OVERSIZE_CELL_CHECK",
          256  +#endif
          257  +#if SQLITE_ENABLE_PREUPDATE_HOOK
          258  +  "ENABLE_PREUPDATE_HOOK",
          259  +#endif
          260  +#if SQLITE_ENABLE_RBU
          261  +  "ENABLE_RBU",
   149    262   #endif
   150    263   #if SQLITE_ENABLE_RTREE
   151    264     "ENABLE_RTREE",
          265  +#endif
          266  +#if SQLITE_ENABLE_SELECTTRACE
          267  +  "ENABLE_SELECTTRACE",
          268  +#endif
          269  +#if SQLITE_ENABLE_SESSION
          270  +  "ENABLE_SESSION",
          271  +#endif
          272  +#if SQLITE_ENABLE_SNAPSHOT
          273  +  "ENABLE_SNAPSHOT",
          274  +#endif
          275  +#if SQLITE_ENABLE_SQLLOG
          276  +  "ENABLE_SQLLOG",
   152    277   #endif
   153    278   #if defined(SQLITE_ENABLE_STAT4)
   154    279     "ENABLE_STAT4",
   155    280   #elif defined(SQLITE_ENABLE_STAT3)
   156    281     "ENABLE_STAT3",
          282  +#endif
          283  +#if SQLITE_ENABLE_STMT_SCANSTATUS
          284  +  "ENABLE_STMT_SCANSTATUS",
          285  +#endif
          286  +#if SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
          287  +  "ENABLE_UNKNOWN_SQL_FUNCTION",
   157    288   #endif
   158    289   #if SQLITE_ENABLE_UNLOCK_NOTIFY
   159    290     "ENABLE_UNLOCK_NOTIFY",
   160    291   #endif
   161    292   #if SQLITE_ENABLE_UPDATE_DELETE_LIMIT
   162    293     "ENABLE_UPDATE_DELETE_LIMIT",
   163    294   #endif
   164         -#if defined(SQLITE_ENABLE_URI_00_ERROR)
          295  +#if SQLITE_ENABLE_URI_00_ERROR
   165    296     "ENABLE_URI_00_ERROR",
          297  +#endif
          298  +#if SQLITE_ENABLE_VFSTRACE
          299  +  "ENABLE_VFSTRACE",
          300  +#endif
          301  +#if SQLITE_ENABLE_WHERETRACE
          302  +  "ENABLE_WHERETRACE",
          303  +#endif
          304  +#if SQLITE_ENABLE_ZIPVFS
          305  +  "ENABLE_ZIPVFS",
          306  +#endif
          307  +#if SQLITE_EXPLAIN_ESTIMATED_ROWS
          308  +  "EXPLAIN_ESTIMATED_ROWS",
          309  +#endif
          310  +#if SQLITE_EXTRA_IFNULLROW
          311  +  "EXTRA_IFNULLROW",
          312  +#endif
          313  +#ifdef SQLITE_EXTRA_INIT
          314  +  "EXTRA_INIT=" CTIMEOPT_VAL(SQLITE_EXTRA_INIT),
          315  +#endif
          316  +#ifdef SQLITE_EXTRA_SHUTDOWN
          317  +  "EXTRA_SHUTDOWN=" CTIMEOPT_VAL(SQLITE_EXTRA_SHUTDOWN),
          318  +#endif
          319  +#ifdef SQLITE_FTS3_MAX_EXPR_DEPTH
          320  +  "FTS3_MAX_EXPR_DEPTH=" CTIMEOPT_VAL(SQLITE_FTS3_MAX_EXPR_DEPTH),
          321  +#endif
          322  +#if SQLITE_FTS5_ENABLE_TEST_MI
          323  +  "FTS5_ENABLE_TEST_MI",
          324  +#endif
          325  +#if SQLITE_FTS5_NO_WITHOUT_ROWID
          326  +  "FTS5_NO_WITHOUT_ROWID",
   166    327   #endif
   167    328   #if SQLITE_HAS_CODEC
   168    329     "HAS_CODEC",
   169    330   #endif
   170    331   #if HAVE_ISNAN || SQLITE_HAVE_ISNAN
   171    332     "HAVE_ISNAN",
   172    333   #endif
................................................................................
   175    336   #endif
   176    337   #if SQLITE_IGNORE_AFP_LOCK_ERRORS
   177    338     "IGNORE_AFP_LOCK_ERRORS",
   178    339   #endif
   179    340   #if SQLITE_IGNORE_FLOCK_LOCK_ERRORS
   180    341     "IGNORE_FLOCK_LOCK_ERRORS",
   181    342   #endif
   182         -#ifdef SQLITE_INT64_TYPE
          343  +#if SQLITE_INLINE_MEMCPY
          344  +  "INLINE_MEMCPY",
          345  +#endif
          346  +#if SQLITE_INT64_TYPE
   183    347     "INT64_TYPE",
   184    348   #endif
   185         -#ifdef SQLITE_LIKE_DOESNT_MATCH_BLOBS
          349  +#ifdef SQLITE_INTEGRITY_CHECK_ERROR_MAX
          350  +  "INTEGRITY_CHECK_ERROR_MAX=" CTIMEOPT_VAL(SQLITE_INTEGRITY_CHECK_ERROR_MAX),
          351  +#endif
          352  +#if SQLITE_LIKE_DOESNT_MATCH_BLOBS
   186    353     "LIKE_DOESNT_MATCH_BLOBS",
   187    354   #endif
   188    355   #if SQLITE_LOCK_TRACE
   189    356     "LOCK_TRACE",
   190    357   #endif
   191         -#if defined(SQLITE_MAX_MMAP_SIZE) && !defined(SQLITE_MAX_MMAP_SIZE_xc)
          358  +#if SQLITE_LOG_CACHE_SPILL
          359  +  "LOG_CACHE_SPILL",
          360  +#endif
          361  +#ifdef SQLITE_MALLOC_SOFT_LIMIT
          362  +  "MALLOC_SOFT_LIMIT=" CTIMEOPT_VAL(SQLITE_MALLOC_SOFT_LIMIT),
          363  +#endif
          364  +#ifdef SQLITE_MAX_ATTACHED
          365  +  "MAX_ATTACHED=" CTIMEOPT_VAL(SQLITE_MAX_ATTACHED),
          366  +#endif
          367  +#ifdef SQLITE_MAX_COLUMN
          368  +  "MAX_COLUMN=" CTIMEOPT_VAL(SQLITE_MAX_COLUMN),
          369  +#endif
          370  +#ifdef SQLITE_MAX_COMPOUND_SELECT
          371  +  "MAX_COMPOUND_SELECT=" CTIMEOPT_VAL(SQLITE_MAX_COMPOUND_SELECT),
          372  +#endif
          373  +#ifdef SQLITE_MAX_DEFAULT_PAGE_SIZE
          374  +  "MAX_DEFAULT_PAGE_SIZE=" CTIMEOPT_VAL(SQLITE_MAX_DEFAULT_PAGE_SIZE),
          375  +#endif
          376  +#ifdef SQLITE_MAX_EXPR_DEPTH
          377  +  "MAX_EXPR_DEPTH=" CTIMEOPT_VAL(SQLITE_MAX_EXPR_DEPTH),
          378  +#endif
          379  +#ifdef SQLITE_MAX_FUNCTION_ARG
          380  +  "MAX_FUNCTION_ARG=" CTIMEOPT_VAL(SQLITE_MAX_FUNCTION_ARG),
          381  +#endif
          382  +#ifdef SQLITE_MAX_LENGTH
          383  +  "MAX_LENGTH=" CTIMEOPT_VAL(SQLITE_MAX_LENGTH),
          384  +#endif
          385  +#ifdef SQLITE_MAX_LIKE_PATTERN_LENGTH
          386  +  "MAX_LIKE_PATTERN_LENGTH=" CTIMEOPT_VAL(SQLITE_MAX_LIKE_PATTERN_LENGTH),
          387  +#endif
          388  +#ifdef SQLITE_MAX_MEMORY
          389  +  "MAX_MEMORY=" CTIMEOPT_VAL(SQLITE_MAX_MEMORY),
          390  +#endif
          391  +#ifdef SQLITE_MAX_MMAP_SIZE
   192    392     "MAX_MMAP_SIZE=" CTIMEOPT_VAL(SQLITE_MAX_MMAP_SIZE),
   193    393   #endif
          394  +#ifdef SQLITE_MAX_MMAP_SIZE_
          395  +  "MAX_MMAP_SIZE_=" CTIMEOPT_VAL(SQLITE_MAX_MMAP_SIZE_),
          396  +#endif
          397  +#ifdef SQLITE_MAX_PAGE_COUNT
          398  +  "MAX_PAGE_COUNT=" CTIMEOPT_VAL(SQLITE_MAX_PAGE_COUNT),
          399  +#endif
          400  +#ifdef SQLITE_MAX_PAGE_SIZE
          401  +  "MAX_PAGE_SIZE=" CTIMEOPT_VAL(SQLITE_MAX_PAGE_SIZE),
          402  +#endif
   194    403   #ifdef SQLITE_MAX_SCHEMA_RETRY
   195    404     "MAX_SCHEMA_RETRY=" CTIMEOPT_VAL(SQLITE_MAX_SCHEMA_RETRY),
   196    405   #endif
          406  +#ifdef SQLITE_MAX_SQL_LENGTH
          407  +  "MAX_SQL_LENGTH=" CTIMEOPT_VAL(SQLITE_MAX_SQL_LENGTH),
          408  +#endif
          409  +#ifdef SQLITE_MAX_TRIGGER_DEPTH
          410  +  "MAX_TRIGGER_DEPTH=" CTIMEOPT_VAL(SQLITE_MAX_TRIGGER_DEPTH),
          411  +#endif
          412  +#ifdef SQLITE_MAX_VARIABLE_NUMBER
          413  +  "MAX_VARIABLE_NUMBER=" CTIMEOPT_VAL(SQLITE_MAX_VARIABLE_NUMBER),
          414  +#endif
          415  +#ifdef SQLITE_MAX_VDBE_OP
          416  +  "MAX_VDBE_OP=" CTIMEOPT_VAL(SQLITE_MAX_VDBE_OP),
          417  +#endif
          418  +#ifdef SQLITE_MAX_WORKER_THREADS
          419  +  "MAX_WORKER_THREADS=" CTIMEOPT_VAL(SQLITE_MAX_WORKER_THREADS),
          420  +#endif
   197    421   #if SQLITE_MEMDEBUG
   198    422     "MEMDEBUG",
   199    423   #endif
   200    424   #if SQLITE_MIXED_ENDIAN_64BIT_FLOAT
   201    425     "MIXED_ENDIAN_64BIT_FLOAT",
          426  +#endif
          427  +#if SQLITE_MMAP_READWRITE
          428  +  "MMAP_READWRITE",
          429  +#endif
          430  +#if SQLITE_MUTEX_NOOP
          431  +  "MUTEX_NOOP",
          432  +#endif
          433  +#if SQLITE_MUTEX_NREF
          434  +  "MUTEX_NREF",
          435  +#endif
          436  +#if SQLITE_MUTEX_OMIT
          437  +  "MUTEX_OMIT",
          438  +#endif
          439  +#if SQLITE_MUTEX_PTHREADS
          440  +  "MUTEX_PTHREADS",
          441  +#endif
          442  +#if SQLITE_MUTEX_W32
          443  +  "MUTEX_W32",
          444  +#endif
          445  +#if SQLITE_NEED_ERR_NAME
          446  +  "NEED_ERR_NAME",
          447  +#endif
          448  +#if SQLITE_NOINLINE
          449  +  "NOINLINE",
   202    450   #endif
   203    451   #if SQLITE_NO_SYNC
   204    452     "NO_SYNC",
   205    453   #endif
   206    454   #if SQLITE_OMIT_ALTERTABLE
   207    455     "OMIT_ALTERTABLE",
   208    456   #endif
................................................................................
   247    495   #endif
   248    496   #if SQLITE_OMIT_COMPLETE
   249    497     "OMIT_COMPLETE",
   250    498   #endif
   251    499   #if SQLITE_OMIT_COMPOUND_SELECT
   252    500     "OMIT_COMPOUND_SELECT",
   253    501   #endif
          502  +#if SQLITE_OMIT_CONFLICT_CLAUSE
          503  +  "OMIT_CONFLICT_CLAUSE",
          504  +#endif
   254    505   #if SQLITE_OMIT_CTE
   255    506     "OMIT_CTE",
   256    507   #endif
   257    508   #if SQLITE_OMIT_DATETIME_FUNCS
   258    509     "OMIT_DATETIME_FUNCS",
   259    510   #endif
   260    511   #if SQLITE_OMIT_DECLTYPE
................................................................................
   276    527     "OMIT_FLOATING_POINT",
   277    528   #endif
   278    529   #if SQLITE_OMIT_FOREIGN_KEY
   279    530     "OMIT_FOREIGN_KEY",
   280    531   #endif
   281    532   #if SQLITE_OMIT_GET_TABLE
   282    533     "OMIT_GET_TABLE",
          534  +#endif
          535  +#if SQLITE_OMIT_HEX_INTEGER
          536  +  "OMIT_HEX_INTEGER",
   283    537   #endif
   284    538   #if SQLITE_OMIT_INCRBLOB
   285    539     "OMIT_INCRBLOB",
   286    540   #endif
   287    541   #if SQLITE_OMIT_INTEGRITY_CHECK
   288    542     "OMIT_INTEGRITY_CHECK",
   289    543   #endif
................................................................................
   303    557     "OMIT_MEMORYDB",
   304    558   #endif
   305    559   #if SQLITE_OMIT_OR_OPTIMIZATION
   306    560     "OMIT_OR_OPTIMIZATION",
   307    561   #endif
   308    562   #if SQLITE_OMIT_PAGER_PRAGMAS
   309    563     "OMIT_PAGER_PRAGMAS",
          564  +#endif
          565  +#if SQLITE_OMIT_PARSER_TRACE
          566  +  "OMIT_PARSER_TRACE",
          567  +#endif
          568  +#if SQLITE_OMIT_POPEN
          569  +  "OMIT_POPEN",
   310    570   #endif
   311    571   #if SQLITE_OMIT_PRAGMA
   312    572     "OMIT_PRAGMA",
   313    573   #endif
   314    574   #if SQLITE_OMIT_PROGRESS_CALLBACK
   315    575     "OMIT_PROGRESS_CALLBACK",
   316    576   #endif
................................................................................
   325    585   #endif
   326    586   #if SQLITE_OMIT_SCHEMA_VERSION_PRAGMAS
   327    587     "OMIT_SCHEMA_VERSION_PRAGMAS",
   328    588   #endif
   329    589   #if SQLITE_OMIT_SHARED_CACHE
   330    590     "OMIT_SHARED_CACHE",
   331    591   #endif
          592  +#if SQLITE_OMIT_SHUTDOWN_DIRECTORIES
          593  +  "OMIT_SHUTDOWN_DIRECTORIES",
          594  +#endif
   332    595   #if SQLITE_OMIT_SUBQUERY
   333    596     "OMIT_SUBQUERY",
   334    597   #endif
   335    598   #if SQLITE_OMIT_TCL_VARIABLE
   336    599     "OMIT_TCL_VARIABLE",
   337    600   #endif
   338    601   #if SQLITE_OMIT_TEMPDB
   339    602     "OMIT_TEMPDB",
          603  +#endif
          604  +#if SQLITE_OMIT_TEST_CONTROL
          605  +  "OMIT_TEST_CONTROL",
   340    606   #endif
   341    607   #if SQLITE_OMIT_TRACE
   342    608     "OMIT_TRACE",
   343    609   #endif
   344    610   #if SQLITE_OMIT_TRIGGER
   345    611     "OMIT_TRIGGER",
   346    612   #endif
................................................................................
   363    629     "OMIT_WAL",
   364    630   #endif
   365    631   #if SQLITE_OMIT_WSD
   366    632     "OMIT_WSD",
   367    633   #endif
   368    634   #if SQLITE_OMIT_XFER_OPT
   369    635     "OMIT_XFER_OPT",
          636  +#endif
          637  +#if SQLITE_PCACHE_SEPARATE_HEADER
          638  +  "PCACHE_SEPARATE_HEADER",
   370    639   #endif
   371    640   #if SQLITE_PERFORMANCE_TRACE
   372    641     "PERFORMANCE_TRACE",
   373    642   #endif
          643  +#if SQLITE_POWERSAFE_OVERWRITE
          644  +  "POWERSAFE_OVERWRITE",
          645  +#endif
          646  +#if SQLITE_PREFER_PROXY_LOCKING
          647  +  "PREFER_PROXY_LOCKING",
          648  +#endif
   374    649   #if SQLITE_PROXY_DEBUG
   375    650     "PROXY_DEBUG",
          651  +#endif
          652  +#if SQLITE_REVERSE_UNORDERED_SELECTS
          653  +  "REVERSE_UNORDERED_SELECTS",
   376    654   #endif
   377    655   #if SQLITE_RTREE_INT_ONLY
   378    656     "RTREE_INT_ONLY",
   379    657   #endif
   380    658   #if SQLITE_SECURE_DELETE
   381    659     "SECURE_DELETE",
   382    660   #endif
   383    661   #if SQLITE_SMALL_STACK
   384    662     "SMALL_STACK",
   385    663   #endif
          664  +#ifdef SQLITE_SORTER_PMASZ
          665  +  "SORTER_PMASZ=" CTIMEOPT_VAL(SQLITE_SORTER_PMASZ),
          666  +#endif
   386    667   #if SQLITE_SOUNDEX
   387    668     "SOUNDEX",
          669  +#endif
          670  +#ifdef SQLITE_STAT4_SAMPLES
          671  +  "STAT4_SAMPLES=" CTIMEOPT_VAL(SQLITE_STAT4_SAMPLES),
          672  +#endif
          673  +#ifdef SQLITE_STMTJRNL_SPILL
          674  +  "STMTJRNL_SPILL=" CTIMEOPT_VAL(SQLITE_STMTJRNL_SPILL),
          675  +#endif
          676  +#if SQLITE_SUBSTR_COMPATIBILITY
          677  +  "SUBSTR_COMPATIBILITY",
   388    678   #endif
   389    679   #if SQLITE_SYSTEM_MALLOC
   390    680     "SYSTEM_MALLOC",
   391    681   #endif
   392    682   #if SQLITE_TCL
   393    683     "TCL",
   394    684   #endif
   395         -#if defined(SQLITE_TEMP_STORE) && !defined(SQLITE_TEMP_STORE_xc)
          685  +#ifdef SQLITE_TEMP_STORE
   396    686     "TEMP_STORE=" CTIMEOPT_VAL(SQLITE_TEMP_STORE),
   397    687   #endif
   398    688   #if SQLITE_TEST
   399    689     "TEST",
   400    690   #endif
   401    691   #if defined(SQLITE_THREADSAFE)
   402    692     "THREADSAFE=" CTIMEOPT_VAL(SQLITE_THREADSAFE),
          693  +#elif defined(THREADSAFE)
          694  +  "THREADSAFE=" CTIMEOPT_VAL(THREADSAFE),
          695  +#else
          696  +  "THREADSAFE=1",
          697  +#endif
          698  +#if SQLITE_UNLINK_AFTER_CLOSE
          699  +  "UNLINK_AFTER_CLOSE",
   403    700   #endif
   404    701   #if SQLITE_UNTESTABLE
   405         -  "UNTESTABLE"
          702  +  "UNTESTABLE",
          703  +#endif
          704  +#if SQLITE_USER_AUTHENTICATION
          705  +  "USER_AUTHENTICATION",
   406    706   #endif
   407    707   #if SQLITE_USE_ALLOCA
   408    708     "USE_ALLOCA",
   409    709   #endif
   410         -#if SQLITE_USER_AUTHENTICATION
   411         -  "USER_AUTHENTICATION",
          710  +#if SQLITE_USE_FCNTL_TRACE
          711  +  "USE_FCNTL_TRACE",
          712  +#endif
          713  +#if SQLITE_USE_URI
          714  +  "USE_URI",
          715  +#endif
          716  +#if SQLITE_VDBE_COVERAGE
          717  +  "VDBE_COVERAGE",
   412    718   #endif
   413    719   #if SQLITE_WIN32_MALLOC
   414    720     "WIN32_MALLOC",
   415    721   #endif
   416    722   #if SQLITE_ZERO_MALLOC
   417         -  "ZERO_MALLOC"
          723  +  "ZERO_MALLOC",
   418    724   #endif
          725  +/* 
          726  +** END CODE GENERATED BY tool/mkctime.tcl 
          727  +*/
   419    728   };
   420    729   
   421         -/*
   422         -** Given the name of a compile-time option, return true if that option
   423         -** was used and false if not.
   424         -**
   425         -** The name can optionally begin with "SQLITE_" but the "SQLITE_" prefix
   426         -** is not required for a match.
   427         -*/
   428         -int sqlite3_compileoption_used(const char *zOptName){
   429         -  int i, n;
   430         -
   431         -#if SQLITE_ENABLE_API_ARMOR
   432         -  if( zOptName==0 ){
   433         -    (void)SQLITE_MISUSE_BKPT;
   434         -    return 0;
   435         -  }
   436         -#endif
   437         -  if( sqlite3StrNICmp(zOptName, "SQLITE_", 7)==0 ) zOptName += 7;
   438         -  n = sqlite3Strlen30(zOptName);
   439         -
   440         -  /* Since ArraySize(azCompileOpt) is normally in single digits, a
   441         -  ** linear search is adequate.  No need for a binary search. */
   442         -  for(i=0; i<ArraySize(azCompileOpt); i++){
   443         -    if( sqlite3StrNICmp(zOptName, azCompileOpt[i], n)==0
   444         -     && sqlite3IsIdChar((unsigned char)azCompileOpt[i][n])==0
   445         -    ){
   446         -      return 1;
   447         -    }
   448         -  }
   449         -  return 0;
   450         -}
   451         -
   452         -/*
   453         -** Return the N-th compile-time option string.  If N is out of range,
   454         -** return a NULL pointer.
   455         -*/
   456         -const char *sqlite3_compileoption_get(int N){
   457         -  if( N>=0 && N<ArraySize(azCompileOpt) ){
   458         -    return azCompileOpt[N];
   459         -  }
   460         -  return 0;
          730  +const char **sqlite3CompileOptions(int *pnOpt){
          731  +  *pnOpt = sizeof(sqlite3azCompileOpt) / sizeof(sqlite3azCompileOpt[0]);
          732  +  return (const char**)sqlite3azCompileOpt;
   461    733   }
   462    734   
   463    735   #endif /* SQLITE_OMIT_COMPILEOPTION_DIAGS */

Changes to src/delete.c.

   349    349   #ifndef SQLITE_OMIT_TRUNCATE_OPTIMIZATION
   350    350     /* Special case: A DELETE without a WHERE clause deletes everything.
   351    351     ** It is easier just to erase the whole table. Prior to version 3.6.5,
   352    352     ** this optimization caused the row change count (the value returned by 
   353    353     ** API function sqlite3_count_changes) to be set incorrectly.
   354    354     **
   355    355     ** The "rcauth==SQLITE_OK" terms is the
   356         -  ** IMPLEMENATION-OF: R-17228-37124 If the action code is SQLITE_DELETE and
          356  +  ** IMPLEMENTATION-OF: R-17228-37124 If the action code is SQLITE_DELETE and
   357    357     ** the callback returns SQLITE_IGNORE then the DELETE operation proceeds but
   358    358     ** the truncate optimization is disabled and all rows are deleted
   359    359     ** individually.
   360    360     */
   361    361     if( rcauth==SQLITE_OK
   362    362      && pWhere==0
   363    363      && !bComplex
................................................................................
   455    455           iKey = ++pParse->nMem;
   456    456           nKey = 0;   /* Zero tells OP_Found to use a composite key */
   457    457           sqlite3VdbeAddOp4(v, OP_MakeRecord, iPk, nPk, iKey,
   458    458               sqlite3IndexAffinityStr(pParse->db, pPk), nPk);
   459    459           sqlite3VdbeAddOp4Int(v, OP_IdxInsert, iEphCur, iKey, iPk, nPk);
   460    460         }else{
   461    461           /* Add the rowid of the row to be deleted to the RowSet */
   462         -        nKey = 1;  /* OP_Seek always uses a single rowid */
          462  +        nKey = 1;  /* OP_DeferredSeek always uses a single rowid */
   463    463           sqlite3VdbeAddOp2(v, OP_RowSetAdd, iRowSet, iKey);
   464    464         }
   465    465       }
   466    466     
   467    467       /* If this DELETE cannot use the ONEPASS strategy, this is the 
   468    468       ** end of the WHERE loop */
   469    469       if( eOnePass!=ONEPASS_OFF ){

Changes to src/expr.c.

  1484   1484       if( pNew==0 ){
  1485   1485         goto no_mem;
  1486   1486       }
  1487   1487       pList = pNew;
  1488   1488       pList->nAlloc *= 2;
  1489   1489     }
  1490   1490     pItem = &pList->a[pList->nExpr++];
  1491         -  memset(pItem, 0, sizeof(*pItem));
         1491  +  assert( offsetof(struct ExprList_item,zName)==sizeof(pItem->pExpr) );
         1492  +  assert( offsetof(struct ExprList_item,pExpr)==0 );
         1493  +  memset(&pItem->zName,0,sizeof(*pItem)-offsetof(struct ExprList_item,zName));
  1492   1494     pItem->pExpr = pExpr;
  1493   1495     return pList;
  1494   1496   
  1495   1497   no_mem:     
  1496   1498     /* Avoid leaking memory if malloc has failed. */
  1497   1499     sqlite3ExprDelete(db, pExpr);
  1498   1500     sqlite3ExprListDelete(db, pList);
................................................................................
  1768   1770   static int selectNodeIsConstant(Walker *pWalker, Select *NotUsed){
  1769   1771     UNUSED_PARAMETER(NotUsed);
  1770   1772     pWalker->eCode = 0;
  1771   1773     return WRC_Abort;
  1772   1774   }
  1773   1775   static int exprIsConst(Expr *p, int initFlag, int iCur){
  1774   1776     Walker w;
  1775         -  memset(&w, 0, sizeof(w));
  1776   1777     w.eCode = initFlag;
  1777   1778     w.xExprCallback = exprNodeIsConstant;
  1778   1779     w.xSelectCallback = selectNodeIsConstant;
         1780  +#ifdef SQLITE_DEBUG
         1781  +  w.xSelectCallback2 = sqlite3SelectWalkAssert2;
         1782  +#endif
  1779   1783     w.u.iCur = iCur;
  1780   1784     sqlite3WalkExpr(&w, p);
  1781   1785     return w.eCode;
  1782   1786   }
  1783   1787   
  1784   1788   /*
  1785   1789   ** Walk an expression tree.  Return non-zero if the expression is constant
................................................................................
  1859   1863   ** sequence as the GROUP BY term, but that is much harder to check,
  1860   1864   ** alternative collating sequences are uncommon, and this is only an
  1861   1865   ** optimization, so we take the easy way out and simply require the
  1862   1866   ** GROUP BY to use the BINARY collating sequence.
  1863   1867   */
  1864   1868   int sqlite3ExprIsConstantOrGroupBy(Parse *pParse, Expr *p, ExprList *pGroupBy){
  1865   1869     Walker w;
  1866         -  memset(&w, 0, sizeof(w));
  1867   1870     w.eCode = 1;
  1868   1871     w.xExprCallback = exprNodeIsConstantOrGroupBy;
         1872  +  w.xSelectCallback = 0;
  1869   1873     w.u.pGroupBy = pGroupBy;
  1870   1874     w.pParse = pParse;
  1871   1875     sqlite3WalkExpr(&w, p);
  1872   1876     return w.eCode;
  1873   1877   }
  1874   1878   
  1875   1879   /*
................................................................................
  1889   1893   #ifdef SQLITE_ENABLE_CURSOR_HINTS
  1890   1894   /*
  1891   1895   ** Walk an expression tree.  Return 1 if the expression contains a
  1892   1896   ** subquery of some kind.  Return 0 if there are no subqueries.
  1893   1897   */
  1894   1898   int sqlite3ExprContainsSubquery(Expr *p){
  1895   1899     Walker w;
  1896         -  memset(&w, 0, sizeof(w));
  1897   1900     w.eCode = 1;
  1898   1901     w.xExprCallback = sqlite3ExprWalkNoop;
  1899   1902     w.xSelectCallback = selectNodeIsConstant;
         1903  +#ifdef SQLITE_DEBUG
         1904  +  w.xSelectCallback2 = sqlite3SelectWalkAssert2;
         1905  +#endif
  1900   1906     sqlite3WalkExpr(&w, p);
  1901   1907     return w.eCode==0;
  1902   1908   }
  1903   1909   #endif
  1904   1910   
  1905   1911   /*
  1906   1912   ** If the expression p codes a constant integer that is small enough
................................................................................
  4897   4903   ** has no arguments or has only constant arguments.  Return false if pExpr
  4898   4904   ** references columns but not columns of tables found in pSrcList.
  4899   4905   */
  4900   4906   int sqlite3FunctionUsesThisSrc(Expr *pExpr, SrcList *pSrcList){
  4901   4907     Walker w;
  4902   4908     struct SrcCount cnt;
  4903   4909     assert( pExpr->op==TK_AGG_FUNCTION );
  4904         -  memset(&w, 0, sizeof(w));
  4905   4910     w.xExprCallback = exprSrcCount;
         4911  +  w.xSelectCallback = 0;
  4906   4912     w.u.pSrcCount = &cnt;
  4907   4913     cnt.pSrc = pSrcList;
  4908   4914     cnt.nThis = 0;
  4909   4915     cnt.nOther = 0;
  4910   4916     sqlite3WalkExprList(&w, pExpr->x.pList);
  4911   4917     return cnt.nThis>0 || cnt.nOther==0;
  4912   4918   }
................................................................................
  5070   5076           return WRC_Continue;
  5071   5077         }
  5072   5078       }
  5073   5079     }
  5074   5080     return WRC_Continue;
  5075   5081   }
  5076   5082   static int analyzeAggregatesInSelect(Walker *pWalker, Select *pSelect){
  5077         -  UNUSED_PARAMETER(pWalker);
  5078   5083     UNUSED_PARAMETER(pSelect);
         5084  +  pWalker->walkerDepth++;
  5079   5085     return WRC_Continue;
         5086  +}
         5087  +static void analyzeAggregatesInSelectEnd(Walker *pWalker, Select *pSelect){
         5088  +  UNUSED_PARAMETER(pSelect);
         5089  +  pWalker->walkerDepth--;
  5080   5090   }
  5081   5091   
  5082   5092   /*
  5083   5093   ** Analyze the pExpr expression looking for aggregate functions and
  5084   5094   ** for variables that need to be added to AggInfo object that pNC->pAggInfo
  5085   5095   ** points to.  Additional entries are made on the AggInfo object as
  5086   5096   ** necessary.
  5087   5097   **
  5088   5098   ** This routine should only be called after the expression has been
  5089   5099   ** analyzed by sqlite3ResolveExprNames().
  5090   5100   */
  5091   5101   void sqlite3ExprAnalyzeAggregates(NameContext *pNC, Expr *pExpr){
  5092   5102     Walker w;
  5093         -  memset(&w, 0, sizeof(w));
  5094   5103     w.xExprCallback = analyzeAggregate;
  5095   5104     w.xSelectCallback = analyzeAggregatesInSelect;
         5105  +  w.xSelectCallback2 = analyzeAggregatesInSelectEnd;
         5106  +  w.walkerDepth = 0;
  5096   5107     w.u.pNC = pNC;
  5097   5108     assert( pNC->pSrcList!=0 );
  5098   5109     sqlite3WalkExpr(&w, pExpr);
  5099   5110   }
  5100   5111   
  5101   5112   /*
  5102   5113   ** Call sqlite3ExprAnalyzeAggregates() for every expression in an

Changes to src/fkey.c.

   629    629     sNameContext.pSrcList = pSrc;
   630    630     sNameContext.pParse = pParse;
   631    631     sqlite3ResolveExprNames(&sNameContext, pWhere);
   632    632   
   633    633     /* Create VDBE to loop through the entries in pSrc that match the WHERE
   634    634     ** clause. For each row found, increment either the deferred or immediate
   635    635     ** foreign key constraint counter. */
   636         -  pWInfo = sqlite3WhereBegin(pParse, pSrc, pWhere, 0, 0, 0, 0);
   637         -  sqlite3VdbeAddOp2(v, OP_FkCounter, pFKey->isDeferred, nIncr);
   638         -  if( pWInfo ){
   639         -    sqlite3WhereEnd(pWInfo);
          636  +  if( pParse->nErr==0 ){
          637  +    pWInfo = sqlite3WhereBegin(pParse, pSrc, pWhere, 0, 0, 0, 0);
          638  +    sqlite3VdbeAddOp2(v, OP_FkCounter, pFKey->isDeferred, nIncr);
          639  +    if( pWInfo ){
          640  +      sqlite3WhereEnd(pWInfo);
          641  +    }
   640    642     }
   641    643   
   642    644     /* Clean up the WHERE clause constructed above. */
   643    645     sqlite3ExprDelete(db, pWhere);
   644    646     if( iFkIfZero ){
   645    647       sqlite3VdbeJumpHere(v, iFkIfZero);
   646    648     }

Changes to src/insert.c.

   517    517   #ifndef SQLITE_OMIT_TRIGGER
   518    518     int isView;                 /* True if attempting to insert into a view */
   519    519     Trigger *pTrigger;          /* List of triggers on pTab, if required */
   520    520     int tmask;                  /* Mask of trigger times */
   521    521   #endif
   522    522   
   523    523     db = pParse->db;
   524         -  memset(&dest, 0, sizeof(dest));
   525    524     if( pParse->nErr || db->mallocFailed ){
   526    525       goto insert_cleanup;
   527    526     }
          527  +  dest.iSDParm = 0;  /* Suppress a harmless compiler warning */
   528    528   
   529    529     /* If the Select object is really just a simple VALUES() list with a
   530    530     ** single row (the common case) then keep that one row of values
   531    531     ** and discard the other (unused) parts of the pSelect object
   532    532     */
   533    533     if( pSelect && (pSelect->selFlags & SF_Values)!=0 && pSelect->pPrior==0 ){
   534    534       pList = pSelect->pEList;

Changes to src/main.c.

   866    866     int nKey2, const void *pKey2
   867    867   ){
   868    868     int rc, n;
   869    869     n = nKey1<nKey2 ? nKey1 : nKey2;
   870    870     /* EVIDENCE-OF: R-65033-28449 The built-in BINARY collation compares
   871    871     ** strings byte by byte using the memcmp() function from the standard C
   872    872     ** library. */
          873  +  assert( pKey1 && pKey2 );
   873    874     rc = memcmp(pKey1, pKey2, n);
   874    875     if( rc==0 ){
   875    876       if( padFlag
   876    877        && allSpaces(((char*)pKey1)+n, nKey1-n)
   877    878        && allSpaces(((char*)pKey2)+n, nKey2-n)
   878    879       ){
   879    880         /* EVIDENCE-OF: R-31624-24737 RTRIM is like BINARY except that extra
................................................................................
  3452   3453     return reportError(SQLITE_MISUSE, lineno, "misuse");
  3453   3454   }
  3454   3455   int sqlite3CantopenError(int lineno){
  3455   3456     testcase( sqlite3GlobalConfig.xLog!=0 );
  3456   3457     return reportError(SQLITE_CANTOPEN, lineno, "cannot open file");
  3457   3458   }
  3458   3459   #ifdef SQLITE_DEBUG
         3460  +int sqlite3CorruptPgnoError(int lineno, Pgno pgno){
         3461  +  char zMsg[100];
         3462  +  sqlite3_snprintf(sizeof(zMsg), zMsg, "database corruption page %d", pgno);
         3463  +  testcase( sqlite3GlobalConfig.xLog!=0 );
         3464  +  return reportError(SQLITE_CORRUPT, lineno, zMsg);
         3465  +}
  3459   3466   int sqlite3NomemError(int lineno){
  3460   3467     testcase( sqlite3GlobalConfig.xLog!=0 );
  3461   3468     return reportError(SQLITE_NOMEM, lineno, "OOM");
  3462   3469   }
  3463   3470   int sqlite3IoerrnomemError(int lineno){
  3464   3471     testcase( sqlite3GlobalConfig.xLog!=0 );
  3465   3472     return reportError(SQLITE_IOERR_NOMEM, lineno, "I/O OOM error");
................................................................................
  4245   4252   /*
  4246   4253   ** Free a snapshot handle obtained from sqlite3_snapshot_get().
  4247   4254   */
  4248   4255   void sqlite3_snapshot_free(sqlite3_snapshot *pSnapshot){
  4249   4256     sqlite3_free(pSnapshot);
  4250   4257   }
  4251   4258   #endif /* SQLITE_ENABLE_SNAPSHOT */
         4259  +
         4260  +#ifndef SQLITE_OMIT_COMPILEOPTION_DIAGS
         4261  +/*
         4262  +** Given the name of a compile-time option, return true if that option
         4263  +** was used and false if not.
         4264  +**
         4265  +** The name can optionally begin with "SQLITE_" but the "SQLITE_" prefix
         4266  +** is not required for a match.
         4267  +*/
         4268  +int sqlite3_compileoption_used(const char *zOptName){
         4269  +  int i, n;
         4270  +  int nOpt;
         4271  +  const char **azCompileOpt;
         4272  + 
         4273  +#if SQLITE_ENABLE_API_ARMOR
         4274  +  if( zOptName==0 ){
         4275  +    (void)SQLITE_MISUSE_BKPT;
         4276  +    return 0;
         4277  +  }
         4278  +#endif
         4279  +
         4280  +  azCompileOpt = sqlite3CompileOptions(&nOpt);
         4281  +
         4282  +  if( sqlite3StrNICmp(zOptName, "SQLITE_", 7)==0 ) zOptName += 7;
         4283  +  n = sqlite3Strlen30(zOptName);
         4284  +
         4285  +  /* Since nOpt is normally in single digits, a linear search is 
         4286  +  ** adequate. No need for a binary search. */
         4287  +  for(i=0; i<nOpt; i++){
         4288  +    if( sqlite3StrNICmp(zOptName, azCompileOpt[i], n)==0
         4289  +     && sqlite3IsIdChar((unsigned char)azCompileOpt[i][n])==0
         4290  +    ){
         4291  +      return 1;
         4292  +    }
         4293  +  }
         4294  +  return 0;
         4295  +}
         4296  +
         4297  +/*
         4298  +** Return the N-th compile-time option string.  If N is out of range,
         4299  +** return a NULL pointer.
         4300  +*/
         4301  +const char *sqlite3_compileoption_get(int N){
         4302  +  int nOpt;
         4303  +  const char **azCompileOpt;
         4304  +  azCompileOpt = sqlite3CompileOptions(&nOpt);
         4305  +  if( N>=0 && N<nOpt ){
         4306  +    return azCompileOpt[N];
         4307  +  }
         4308  +  return 0;
         4309  +}
         4310  +#endif /* SQLITE_OMIT_COMPILEOPTION_DIAGS */

Changes to src/pager.c.

  3257   3257     assert( pPager->eState==PAGER_OPEN );
  3258   3258     assert( pPager->eLock>=SHARED_LOCK );
  3259   3259     assert( isOpen(pPager->fd) );
  3260   3260     assert( pPager->tempFile==0 );
  3261   3261     nPage = sqlite3WalDbsize(pPager->pWal);
  3262   3262   
  3263   3263     /* If the number of pages in the database is not available from the
  3264         -  ** WAL sub-system, determine the page counte based on the size of
         3264  +  ** WAL sub-system, determine the page count based on the size of
  3265   3265     ** the database file.  If the size of the database file is not an
  3266   3266     ** integer multiple of the page-size, round up the result.
  3267   3267     */
  3268   3268     if( nPage==0 && ALWAYS(isOpen(pPager->fd)) ){
  3269   3269       i64 n = 0;                    /* Size of db file in bytes */
  3270   3270       int rc = sqlite3OsFileSize(pPager->fd, &n);
  3271   3271       if( rc!=SQLITE_OK ){
................................................................................
  3308   3308   static int pagerOpenWalIfPresent(Pager *pPager){
  3309   3309     int rc = SQLITE_OK;
  3310   3310     assert( pPager->eState==PAGER_OPEN );
  3311   3311     assert( pPager->eLock>=SHARED_LOCK );
  3312   3312   
  3313   3313     if( !pPager->tempFile ){
  3314   3314       int isWal;                    /* True if WAL file exists */
  3315         -    Pgno nPage;                   /* Size of the database file */
  3316         -
  3317         -    rc = pagerPagecount(pPager, &nPage);
  3318         -    if( rc ) return rc;
  3319         -    if( nPage==0 ){
  3320         -      rc = sqlite3OsDelete(pPager->pVfs, pPager->zWal, 0);
  3321         -      if( rc==SQLITE_IOERR_DELETE_NOENT ) rc = SQLITE_OK;
  3322         -      isWal = 0;
  3323         -    }else{
  3324         -      rc = sqlite3OsAccess(
  3325         -          pPager->pVfs, pPager->zWal, SQLITE_ACCESS_EXISTS, &isWal
  3326         -      );
  3327         -    }
         3315  +    rc = sqlite3OsAccess(
         3316  +        pPager->pVfs, pPager->zWal, SQLITE_ACCESS_EXISTS, &isWal
         3317  +    );
  3328   3318       if( rc==SQLITE_OK ){
  3329   3319         if( isWal ){
  3330         -        testcase( sqlite3PcachePagecount(pPager->pPCache)==0 );
  3331         -        rc = sqlite3PagerOpenWal(pPager, 0);
         3320  +        Pgno nPage;                   /* Size of the database file */
         3321  +
         3322  +        rc = pagerPagecount(pPager, &nPage);
         3323  +        if( rc ) return rc;
         3324  +        if( nPage==0 ){
         3325  +          rc = sqlite3OsDelete(pPager->pVfs, pPager->zWal, 0);
         3326  +        }else{
         3327  +          testcase( sqlite3PcachePagecount(pPager->pPCache)==0 );
         3328  +          rc = sqlite3PagerOpenWal(pPager, 0);
         3329  +        }
  3332   3330         }else if( pPager->journalMode==PAGER_JOURNALMODE_WAL ){
  3333   3331           pPager->journalMode = PAGER_JOURNALMODE_DELETE;
  3334   3332         }
  3335   3333       }
  3336   3334     }
  3337   3335     return rc;
  3338   3336   }
................................................................................
  5282   5280         ** other bytes change randomly with each file change when
  5283   5281         ** a codec is in use.
  5284   5282         ** 
  5285   5283         ** There is a vanishingly small chance that a change will not be 
  5286   5284         ** detected.  The chance of an undetected change is so small that
  5287   5285         ** it can be neglected.
  5288   5286         */
  5289         -      Pgno nPage = 0;
  5290   5287         char dbFileVers[sizeof(pPager->dbFileVers)];
  5291   5288   
  5292         -      rc = pagerPagecount(pPager, &nPage);
  5293         -      if( rc ) goto failed;
  5294         -
  5295         -      if( nPage>0 ){
  5296         -        IOTRACE(("CKVERS %p %d\n", pPager, sizeof(dbFileVers)));
  5297         -        rc = sqlite3OsRead(pPager->fd, &dbFileVers, sizeof(dbFileVers), 24);
  5298         -        if( rc!=SQLITE_OK && rc!=SQLITE_IOERR_SHORT_READ ){
         5289  +      IOTRACE(("CKVERS %p %d\n", pPager, sizeof(dbFileVers)));
         5290  +      rc = sqlite3OsRead(pPager->fd, &dbFileVers, sizeof(dbFileVers), 24);
         5291  +      if( rc!=SQLITE_OK ){
         5292  +        if( rc!=SQLITE_IOERR_SHORT_READ ){
  5299   5293             goto failed;
  5300   5294           }
  5301         -      }else{
  5302   5295           memset(dbFileVers, 0, sizeof(dbFileVers));
  5303   5296         }
  5304   5297   
  5305   5298         if( memcmp(pPager->dbFileVers, dbFileVers, sizeof(dbFileVers))!=0 ){
  5306   5299           pager_reset(pPager);
  5307   5300   
  5308   5301           /* Unmap the database file. It is possible that external processes

Changes to src/pcache.h.

    22     22   ** Every page in the cache is controlled by an instance of the following
    23     23   ** structure.
    24     24   */
    25     25   struct PgHdr {
    26     26     sqlite3_pcache_page *pPage;    /* Pcache object page handle */
    27     27     void *pData;                   /* Page data */
    28     28     void *pExtra;                  /* Extra content */
           29  +  PCache *pCache;                /* PRIVATE: Cache that owns this page */
    29     30     PgHdr *pDirty;                 /* Transient list of dirty sorted by pgno */
    30     31     Pager *pPager;                 /* The pager this page is part of */
    31     32     Pgno pgno;                     /* Page number for this page */
    32     33   #ifdef SQLITE_CHECK_PAGES
    33     34     u32 pageHash;                  /* Hash of page content */
    34     35   #endif
    35     36     u16 flags;                     /* PGHDR flags defined below */
    36     37   
    37     38     /**********************************************************************
    38         -  ** Elements above are public.  All that follows is private to pcache.c
    39         -  ** and should not be accessed by other modules.
           39  +  ** Elements above, except pCache, are public.  All that follow are 
           40  +  ** private to pcache.c and should not be accessed by other modules.
           41  +  ** pCache is grouped with the public elements for efficiency.
    40     42     */
    41     43     i16 nRef;                      /* Number of users of this page */
    42         -  PCache *pCache;                /* Cache that owns this page */
    43         -
    44     44     PgHdr *pDirtyNext;             /* Next element in list of dirty pages */
    45     45     PgHdr *pDirtyPrev;             /* Previous element in list of dirty pages */
    46     46   };
    47     47   
    48     48   /* Bit values for PgHdr.flags */
    49     49   #define PGHDR_CLEAN           0x001  /* Page not on the PCache.pDirty list */
    50     50   #define PGHDR_DIRTY           0x002  /* Page is on the PCache.pDirty list */

Changes to src/pragma.h.

   263    263     /* ePragFlg:  */ PragFlg_Result0|PragFlg_NoColumns1,
   264    264     /* ColNames:  */ 0, 0,
   265    265     /* iArg:      */ 0 },
   266    266   #endif
   267    267   #if !defined(SQLITE_OMIT_FOREIGN_KEY) && !defined(SQLITE_OMIT_TRIGGER)
   268    268    {/* zName:     */ "foreign_key_check",
   269    269     /* ePragTyp:  */ PragTyp_FOREIGN_KEY_CHECK,
   270         -  /* ePragFlg:  */ PragFlg_NeedSchema,
          270  +  /* ePragFlg:  */ PragFlg_NeedSchema|PragFlg_Result0,
   271    271     /* ColNames:  */ 39, 4,
   272    272     /* iArg:      */ 0 },
   273    273   #endif
   274    274   #if !defined(SQLITE_OMIT_FOREIGN_KEY)
   275    275    {/* zName:     */ "foreign_key_list",
   276    276     /* ePragTyp:  */ PragTyp_FOREIGN_KEY_LIST,
   277    277     /* ePragFlg:  */ PragFlg_NeedSchema|PragFlg_Result1|PragFlg_SchemaOpt,
................................................................................
   350    350     /* ePragFlg:  */ PragFlg_NeedSchema|PragFlg_Result1|PragFlg_SchemaOpt,
   351    351     /* ColNames:  */ 15, 6,
   352    352     /* iArg:      */ 1 },
   353    353   #endif
   354    354   #if !defined(SQLITE_OMIT_INTEGRITY_CHECK)
   355    355    {/* zName:     */ "integrity_check",
   356    356     /* ePragTyp:  */ PragTyp_INTEGRITY_CHECK,
   357         -  /* ePragFlg:  */ PragFlg_NeedSchema,
          357  +  /* ePragFlg:  */ PragFlg_NeedSchema|PragFlg_Result0|PragFlg_Result1,
   358    358     /* ColNames:  */ 0, 0,
   359    359     /* iArg:      */ 0 },
   360    360   #endif
   361    361   #if !defined(SQLITE_OMIT_PAGER_PRAGMAS)
   362    362    {/* zName:     */ "journal_mode",
   363    363     /* ePragTyp:  */ PragTyp_JOURNAL_MODE,
   364    364     /* ePragFlg:  */ PragFlg_NeedSchema|PragFlg_Result0|PragFlg_SchemaReq,
................................................................................
   445    445     /* ePragFlg:  */ PragFlg_Result0|PragFlg_NoColumns1,
   446    446     /* ColNames:  */ 0, 0,
   447    447     /* iArg:      */ SQLITE_QueryOnly },
   448    448   #endif
   449    449   #if !defined(SQLITE_OMIT_INTEGRITY_CHECK)
   450    450    {/* zName:     */ "quick_check",
   451    451     /* ePragTyp:  */ PragTyp_INTEGRITY_CHECK,
   452         -  /* ePragFlg:  */ PragFlg_NeedSchema,
          452  +  /* ePragFlg:  */ PragFlg_NeedSchema|PragFlg_Result0|PragFlg_Result1,
   453    453     /* ColNames:  */ 0, 0,
   454    454     /* iArg:      */ 0 },
   455    455   #endif
   456    456   #if !defined(SQLITE_OMIT_FLAG_PRAGMAS)
   457    457    {/* zName:     */ "read_uncommitted",
   458    458     /* ePragTyp:  */ PragTyp_FLAG,
   459    459     /* ePragFlg:  */ PragFlg_Result0|PragFlg_NoColumns1,

Changes to src/resolve.c.

  1426   1426   int sqlite3ResolveExprNames( 
  1427   1427     NameContext *pNC,       /* Namespace to resolve expressions in. */
  1428   1428     Expr *pExpr             /* The expression to be analyzed. */
  1429   1429   ){
  1430   1430     u16 savedHasAgg;
  1431   1431     Walker w;
  1432   1432   
  1433         -  if( pExpr==0 ) return 0;
  1434         -#if SQLITE_MAX_EXPR_DEPTH>0
  1435         -  {
  1436         -    Parse *pParse = pNC->pParse;
  1437         -    if( sqlite3ExprCheckHeight(pParse, pExpr->nHeight+pNC->pParse->nHeight) ){
  1438         -      return 1;
  1439         -    }
  1440         -    pParse->nHeight += pExpr->nHeight;
  1441         -  }
  1442         -#endif
         1433  +  if( pExpr==0 ) return SQLITE_OK;
  1443   1434     savedHasAgg = pNC->ncFlags & (NC_HasAgg|NC_MinMaxAgg);
  1444   1435     pNC->ncFlags &= ~(NC_HasAgg|NC_MinMaxAgg);
  1445   1436     w.pParse = pNC->pParse;
  1446   1437     w.xExprCallback = resolveExprStep;
  1447   1438     w.xSelectCallback = resolveSelectStep;
  1448   1439     w.xSelectCallback2 = 0;
  1449         -  w.walkerDepth = 0;
  1450         -  w.eCode = 0;
  1451   1440     w.u.pNC = pNC;
         1441  +#if SQLITE_MAX_EXPR_DEPTH>0
         1442  +  w.pParse->nHeight += pExpr->nHeight;
         1443  +  if( sqlite3ExprCheckHeight(w.pParse, w.pParse->nHeight) ){
         1444  +    return SQLITE_ERROR;
         1445  +  }
         1446  +#endif
  1452   1447     sqlite3WalkExpr(&w, pExpr);
  1453   1448   #if SQLITE_MAX_EXPR_DEPTH>0
  1454         -  pNC->pParse->nHeight -= pExpr->nHeight;
         1449  +  w.pParse->nHeight -= pExpr->nHeight;
  1455   1450   #endif
  1456         -  if( pNC->nErr>0 || w.pParse->nErr>0 ){
  1457         -    ExprSetProperty(pExpr, EP_Error);
  1458         -  }
  1459   1451     if( pNC->ncFlags & NC_HasAgg ){
  1460   1452       ExprSetProperty(pExpr, EP_Agg);
  1461   1453     }
  1462   1454     pNC->ncFlags |= savedHasAgg;
  1463         -  return ExprHasProperty(pExpr, EP_Error);
         1455  +  return pNC->nErr>0 || w.pParse->nErr>0;
  1464   1456   }
  1465   1457   
  1466   1458   /*
  1467   1459   ** Resolve all names for all expression in an expression list.  This is
  1468   1460   ** just like sqlite3ResolveExprNames() except that it works for an expression
  1469   1461   ** list rather than a single expression.
  1470   1462   */
................................................................................
  1497   1489     Parse *pParse,         /* The parser context */
  1498   1490     Select *p,             /* The SELECT statement being coded. */
  1499   1491     NameContext *pOuterNC  /* Name context for parent SELECT statement */
  1500   1492   ){
  1501   1493     Walker w;
  1502   1494   
  1503   1495     assert( p!=0 );
  1504         -  memset(&w, 0, sizeof(w));
  1505   1496     w.xExprCallback = resolveExprStep;
  1506   1497     w.xSelectCallback = resolveSelectStep;
         1498  +  w.xSelectCallback2 = 0;
  1507   1499     w.pParse = pParse;
  1508   1500     w.u.pNC = pOuterNC;
  1509   1501     sqlite3WalkSelect(&w, p);
  1510   1502   }
  1511   1503   
  1512   1504   /*
  1513   1505   ** Resolve names in expressions that can only reference a single table:

Changes to src/select.c.

  1027   1027   }
  1028   1028   
  1029   1029   /*
  1030   1030   ** Allocate a KeyInfo object sufficient for an index of N key columns and
  1031   1031   ** X extra columns.
  1032   1032   */
  1033   1033   KeyInfo *sqlite3KeyInfoAlloc(sqlite3 *db, int N, int X){
  1034         -  int nExtra = (N+X)*(sizeof(CollSeq*)+1);
         1034  +  int nExtra = (N+X)*(sizeof(CollSeq*)+1) - sizeof(CollSeq*);
  1035   1035     KeyInfo *p = sqlite3DbMallocRawNN(db, sizeof(KeyInfo) + nExtra);
  1036   1036     if( p ){
  1037   1037       p->aSortOrder = (u8*)&p->aColl[N+X];
  1038   1038       p->nField = (u16)N;
  1039   1039       p->nXField = (u16)X;
  1040   1040       p->enc = ENC(db);
  1041   1041       p->db = db;
................................................................................
  3201   3201             memset(&ifNullRow, 0, sizeof(ifNullRow));
  3202   3202             ifNullRow.op = TK_IF_NULL_ROW;
  3203   3203             ifNullRow.pLeft = pCopy;
  3204   3204             ifNullRow.iTable = pSubst->iNewTable;
  3205   3205             pCopy = &ifNullRow;
  3206   3206           }
  3207   3207           pNew = sqlite3ExprDup(db, pCopy, 0);
  3208         -        if( pNew && (pExpr->flags & EP_FromJoin) ){
         3208  +        if( pNew && pSubst->isLeftJoin ){
         3209  +          ExprSetProperty(pNew, EP_CanBeNull);
         3210  +        }
         3211  +        if( pNew && ExprHasProperty(pExpr,EP_FromJoin) ){
  3209   3212             pNew->iRightJoinTable = pExpr->iRightJoinTable;
  3210         -          pNew->flags |= EP_FromJoin;
         3213  +          ExprSetProperty(pNew, EP_FromJoin);
  3211   3214           }
  3212   3215           sqlite3ExprDelete(db, pExpr);
  3213   3216           pExpr = pNew;
  3214   3217         }
  3215   3218       }
  3216   3219     }else{
  3217   3220       if( pExpr->op==TK_IF_NULL_ROW && pExpr->iTable==pSubst->iTable ){
................................................................................
  3496   3499     **
  3497   3500     **         (t1 LEFT OUTER JOIN t2) JOIN t3
  3498   3501     **
  3499   3502     ** which is not at all the same thing.
  3500   3503     **
  3501   3504     ** If the subquery is the right operand of a LEFT JOIN, then the outer
  3502   3505     ** query cannot be an aggregate.  This is an artifact of the way aggregates
  3503         -  ** are processed - there is not mechanism to determine if the LEFT JOIN
         3506  +  ** are processed - there is no mechanism to determine if the LEFT JOIN
  3504   3507     ** table should be all-NULL.
  3505   3508     **
  3506   3509     ** See also tickets #306, #350, and #3300.
  3507   3510     */
  3508   3511     if( (pSubitem->fg.jointype & JT_OUTER)!=0 ){
  3509   3512       isLeftJoin = 1;
  3510   3513       if( pSubSrc->nSrc>1 || isAgg ){
................................................................................
  4606   4609   ** subquery in the parser tree.
  4607   4610   */
  4608   4611   int sqlite3ExprWalkNoop(Walker *NotUsed, Expr *NotUsed2){
  4609   4612     UNUSED_PARAMETER2(NotUsed, NotUsed2);
  4610   4613     return WRC_Continue;
  4611   4614   }
  4612   4615   
         4616  +/*
         4617  +** No-op routine for the parse-tree walker for SELECT statements.
         4618  +** subquery in the parser tree.
         4619  +*/
         4620  +int sqlite3SelectWalkNoop(Walker *NotUsed, Select *NotUsed2){
         4621  +  UNUSED_PARAMETER2(NotUsed, NotUsed2);
         4622  +  return WRC_Continue;
         4623  +}
         4624  +
         4625  +#if SQLITE_DEBUG
         4626  +/*
         4627  +** Always assert.  This xSelectCallback2 implementation proves that the
         4628  +** xSelectCallback2 is never invoked.
         4629  +*/
         4630  +void sqlite3SelectWalkAssert2(Walker *NotUsed, Select *NotUsed2){
         4631  +  UNUSED_PARAMETER2(NotUsed, NotUsed2);
         4632  +  assert( 0 );
         4633  +}
         4634  +#endif
  4613   4635   /*
  4614   4636   ** This routine "expands" a SELECT statement and all of its subqueries.
  4615   4637   ** For additional information on what it means to "expand" a SELECT
  4616   4638   ** statement, see the comment on the selectExpand worker callback above.
  4617   4639   **
  4618   4640   ** Expanding a SELECT statement is the first step in processing a
  4619   4641   ** SELECT statement.  The SELECT statement must be expanded before
................................................................................
  4621   4643   **
  4622   4644   ** If anything goes wrong, an error message is written into pParse.
  4623   4645   ** The calling function can detect the problem by looking at pParse->nErr
  4624   4646   ** and/or pParse->db->mallocFailed.
  4625   4647   */
  4626   4648   static void sqlite3SelectExpand(Parse *pParse, Select *pSelect){
  4627   4649     Walker w;
  4628         -  memset(&w, 0, sizeof(w));
  4629   4650     w.xExprCallback = sqlite3ExprWalkNoop;
  4630   4651     w.pParse = pParse;
  4631   4652     if( pParse->hasCompound ){
  4632   4653       w.xSelectCallback = convertCompoundSelectToSubquery;
         4654  +    w.xSelectCallback2 = 0;
  4633   4655       sqlite3WalkSelect(&w, pSelect);
  4634   4656     }
  4635   4657     w.xSelectCallback = selectExpander;
  4636   4658     w.xSelectCallback2 = selectPopWith;
  4637   4659     sqlite3WalkSelect(&w, pSelect);
  4638   4660   }
  4639   4661   
................................................................................
  4685   4707   ** SELECT statement.
  4686   4708   **
  4687   4709   ** Use this routine after name resolution.
  4688   4710   */
  4689   4711   static void sqlite3SelectAddTypeInfo(Parse *pParse, Select *pSelect){
  4690   4712   #ifndef SQLITE_OMIT_SUBQUERY
  4691   4713     Walker w;
  4692         -  memset(&w, 0, sizeof(w));
         4714  +  w.xSelectCallback = sqlite3SelectWalkNoop;
  4693   4715     w.xSelectCallback2 = selectAddSubqueryTypeInfo;
  4694   4716     w.xExprCallback = sqlite3ExprWalkNoop;
  4695   4717     w.pParse = pParse;
  4696   4718     sqlite3WalkSelect(&w, pSelect);
  4697   4719   #endif
  4698   4720   }
  4699   4721   
................................................................................
  5266   5288           VdbeComment((v, "materialize \"%s\"", pItem->pTab->zName));
  5267   5289         }else{
  5268   5290           VdbeNoopComment((v, "materialize \"%s\"", pItem->pTab->zName));
  5269   5291         }
  5270   5292         pPrior = isSelfJoinView(pTabList, pItem);
  5271   5293         if( pPrior ){
  5272   5294           sqlite3VdbeAddOp2(v, OP_OpenDup, pItem->iCursor, pPrior->iCursor);
         5295  +        explainSetInteger(pItem->iSelectId, pPrior->iSelectId);
         5296  +        assert( pPrior->pSelect!=0 );
         5297  +        pSub->nSelectRow = pPrior->pSelect->nSelectRow;
  5273   5298         }else{
  5274   5299           sqlite3SelectDestInit(&dest, SRT_EphemTab, pItem->iCursor);
  5275   5300           explainSetInteger(pItem->iSelectId, (u8)pParse->iNextSelectId);
  5276   5301           sqlite3Select(pParse, pSub, &dest);
  5277   5302         }
  5278   5303         pItem->pTab->nRowLogEst = pSub->nSelectRow;
  5279   5304         if( onceAddr ) sqlite3VdbeJumpHere(v, onceAddr);

Changes to src/shell.c.

    14     14   */
    15     15   #if (defined(_WIN32) || defined(WIN32)) && !defined(_CRT_SECURE_NO_WARNINGS)
    16     16   /* This needs to come before any includes for MSVC compiler */
    17     17   #define _CRT_SECURE_NO_WARNINGS
    18     18   #endif
    19     19   
    20     20   /*
    21         -** If requested, include the SQLite compiler options file for MSVC.
           21  +** Warning pragmas copied from msvc.h in the core.
    22     22   */
    23         -#if defined(INCLUDE_MSVC_H)
    24         -#include "msvc.h"
    25         -#endif
           23  +#if defined(_MSC_VER)
           24  +#pragma warning(disable : 4054)
           25  +#pragma warning(disable : 4055)
           26  +#pragma warning(disable : 4100)
           27  +#pragma warning(disable : 4127)
           28  +#pragma warning(disable : 4130)
           29  +#pragma warning(disable : 4152)
           30  +#pragma warning(disable : 4189)
           31  +#pragma warning(disable : 4206)
           32  +#pragma warning(disable : 4210)
           33  +#pragma warning(disable : 4232)
           34  +#pragma warning(disable : 4244)
           35  +#pragma warning(disable : 4305)
           36  +#pragma warning(disable : 4306)
           37  +#pragma warning(disable : 4702)
           38  +#pragma warning(disable : 4706)
           39  +#endif /* defined(_MSC_VER) */
    26     40   
    27     41   /*
    28     42   ** No support for loadable extensions in VxWorks.
    29     43   */
    30     44   #if (defined(__RTP__) || defined(_WRS_KERNEL)) && !SQLITE_OMIT_LOAD_EXTENSION
    31     45   # define SQLITE_OMIT_LOAD_EXTENSION 1
    32     46   #endif
................................................................................
   490    504   ** lower 30 bits of a 32-bit signed integer.
   491    505   */
   492    506   static int strlen30(const char *z){
   493    507     const char *z2 = z;
   494    508     while( *z2 ){ z2++; }
   495    509     return 0x3fffffff & (int)(z2 - z);
   496    510   }
          511  +
          512  +/*
          513  +** Return the length of a string in characters.  Multibyte UTF8 characters
          514  +** count as a single character.
          515  +*/
          516  +static int strlenChar(const char *z){
          517  +  int n = 0;
          518  +  while( *z ){
          519  +    if( (0xc0&*(z++))!=0x80 ) n++;
          520  +  }
          521  +  return n;
          522  +}
   497    523   
   498    524   /*
   499    525   ** This routine reads a line of text from FILE in, stores
   500    526   ** the text in memory obtained from malloc() and returns a pointer
   501    527   ** to the text.  NULL is returned at end of file, or if malloc()
   502    528   ** fails.
   503    529   **
................................................................................
   698    724         lwr = mid+1;
   699    725       }else{
   700    726         upr = mid-1;
   701    727       }
   702    728     }
   703    729     return 0;
   704    730   }
          731  +
          732  +/*
          733  +** SQL function:  shell_add_schema(S,X)
          734  +**
          735  +** Add the schema name X to the CREATE statement in S and return the result.
          736  +** Examples:
          737  +**
          738  +**    CREATE TABLE t1(x)   ->   CREATE TABLE xyz.t1(x);
          739  +**
          740  +** Also works on
          741  +**
          742  +**    CREATE INDEX
          743  +**    CREATE UNIQUE INDEX
          744  +**    CREATE VIEW
          745  +**    CREATE TRIGGER
          746  +**    CREATE VIRTUAL TABLE
          747  +**
          748  +** This UDF is used by the .schema command to insert the schema name of
          749  +** attached databases into the middle of the sqlite_master.sql field.
          750  +*/
          751  +static void shellAddSchemaName(
          752  +  sqlite3_context *pCtx,
          753  +  int nVal,
          754  +  sqlite3_value **apVal
          755  +){
          756  +  static const char *aPrefix[] = {
          757  +     "TABLE",
          758  +     "INDEX",
          759  +     "UNIQUE INDEX",
          760  +     "VIEW",
          761  +     "TRIGGER",
          762  +     "VIRTUAL TABLE"
          763  +  };
          764  +  int i = 0;
          765  +  const char *zIn = (const char*)sqlite3_value_text(apVal[0]);
          766  +  const char *zSchema = (const char*)sqlite3_value_text(apVal[1]);
          767  +  assert( nVal==2 );
          768  +  if( zIn!=0 && strncmp(zIn, "CREATE ", 7)==0 ){
          769  +    for(i=0; i<sizeof(aPrefix)/sizeof(aPrefix[0]); i++){
          770  +      int n = strlen30(aPrefix[i]);
          771  +      if( strncmp(zIn+7, aPrefix[i], n)==0 && zIn[n+7]==' ' ){
          772  +        char cQuote = quoteChar(zSchema);
          773  +        char *z;
          774  +        if( cQuote ){
          775  +         z = sqlite3_mprintf("%.*s \"%w\".%s", n+7, zIn, zSchema, zIn+n+8);
          776  +        }else{
          777  +          z = sqlite3_mprintf("%.*s %s.%s", n+7, zIn, zSchema, zIn+n+8);
          778  +        }
          779  +        sqlite3_result_text(pCtx, z, -1, sqlite3_free);
          780  +        return;
          781  +      }
          782  +    }
          783  +  }
          784  +  sqlite3_result_value(pCtx, apVal[0]);
          785  +}
   705    786   
   706    787   /******************************************************************************
   707    788   ** SHA3 hash implementation copied from ../ext/misc/shathree.c
   708    789   */
   709    790   typedef sqlite3_uint64 u64;
   710    791   /*
   711    792   ** Macros to determine whether the machine is big or little endian,
................................................................................
  1159   1240     return &p->u.x[p->nRate];
  1160   1241   }
  1161   1242   
  1162   1243   /*
  1163   1244   ** Implementation of the sha3(X,SIZE) function.
  1164   1245   **
  1165   1246   ** Return a BLOB which is the SIZE-bit SHA3 hash of X.  The default
  1166         -** size is 256.  If X is a BLOB, it is hashed as is.  
         1247  +** size is 256.  If X is a BLOB, it is hashed as is.
  1167   1248   ** For all other non-NULL types of input, X is converted into a UTF-8 string
  1168   1249   ** and the string is hashed without the trailing 0x00 terminator.  The hash
  1169   1250   ** of a NULL value is NULL.
  1170   1251   */
  1171   1252   static void sha3Func(
  1172   1253     sqlite3_context *context,
  1173   1254     int argc,
................................................................................
  1899   1980             int w, n;
  1900   1981             if( i<ArraySize(p->colWidth) ){
  1901   1982               w = colWidth[i];
  1902   1983             }else{
  1903   1984               w = 0;
  1904   1985             }
  1905   1986             if( w==0 ){
  1906         -            w = strlen30(azCol[i] ? azCol[i] : "");
         1987  +            w = strlenChar(azCol[i] ? azCol[i] : "");
  1907   1988               if( w<10 ) w = 10;
  1908         -            n = strlen30(azArg && azArg[i] ? azArg[i] : p->nullValue);
         1989  +            n = strlenChar(azArg && azArg[i] ? azArg[i] : p->nullValue);
  1909   1990               if( w<n ) w = n;
  1910   1991             }
  1911   1992             if( i<ArraySize(p->actualWidth) ){
  1912   1993               p->actualWidth[i] = w;
  1913   1994             }
  1914   1995             if( showHdr ){
  1915   1996               utf8_width_print(p->out, w, azCol[i]);
................................................................................
  1936   2017         for(i=0; i<nArg; i++){
  1937   2018           int w;
  1938   2019           if( i<ArraySize(p->actualWidth) ){
  1939   2020              w = p->actualWidth[i];
  1940   2021           }else{
  1941   2022              w = 10;
  1942   2023           }
  1943         -        if( p->cMode==MODE_Explain && azArg[i] && strlen30(azArg[i])>w ){
  1944         -          w = strlen30(azArg[i]);
         2024  +        if( p->cMode==MODE_Explain && azArg[i] && strlenChar(azArg[i])>w ){
         2025  +          w = strlenChar(azArg[i]);
  1945   2026           }
  1946   2027           if( i==1 && p->aiIndent && p->pStmt ){
  1947   2028             if( p->iIndent<p->nIndent ){
  1948   2029               utf8_printf(p->out, "%*.s", p->aiIndent[p->iIndent], "");
  1949   2030             }
  1950   2031             p->iIndent++;
  1951   2032           }
................................................................................
  2993   3074         }
  2994   3075       }
  2995   3076       azCol[++nCol] = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 1));
  2996   3077       if( sqlite3_column_int(pStmt, 5) ){
  2997   3078         nPK++;
  2998   3079         if( nPK==1
  2999   3080          && sqlite3_stricmp((const char*)sqlite3_column_text(pStmt,2),
  3000         -                          "INTEGER")==0 
         3081  +                          "INTEGER")==0
  3001   3082         ){
  3002   3083           isIPK = 1;
  3003   3084         }else{
  3004   3085           isIPK = 0;
  3005   3086         }
  3006   3087       }
  3007   3088     }
................................................................................
  3230   3311   static char zHelp[] =
  3231   3312   #ifndef SQLITE_OMIT_AUTHORIZATION
  3232   3313     ".auth ON|OFF           Show authorizer callbacks\n"
  3233   3314   #endif
  3234   3315     ".backup ?DB? FILE      Backup DB (default \"main\") to FILE\n"
  3235   3316     ".bail on|off           Stop after hitting an error.  Default OFF\n"
  3236   3317     ".binary on|off         Turn binary output on or off.  Default OFF\n"
         3318  +  ".cd DIRECTORY          Change the working directory to DIRECTORY\n"
  3237   3319     ".changes on|off        Show number of rows changed by SQL\n"
  3238   3320     ".check GLOB            Fail if output since .testcase does not match\n"
  3239   3321     ".clone NEWDB           Clone data into NEWDB from the existing database\n"
  3240   3322     ".databases             List names and files of attached databases\n"
  3241   3323     ".dbinfo ?DB?           Show status information about the database\n"
  3242   3324     ".dump ?TABLE? ...      Dump the database in an SQL text format\n"
  3243   3325     "                         If TABLE specified, only dump tables matching\n"
................................................................................
  3275   3357     "                         line     One value per line\n"
  3276   3358     "                         list     Values delimited by \"|\"\n"
  3277   3359     "                         quote    Escape answers as for SQL\n"
  3278   3360     "                         tabs     Tab-separated values\n"
  3279   3361     "                         tcl      TCL list elements\n"
  3280   3362     ".nullvalue STRING      Use STRING in place of NULL values\n"
  3281   3363     ".once FILENAME         Output for the next SQL command only to FILENAME\n"
  3282         -  ".open ?--new? ?FILE?   Close existing database and reopen FILE\n"
  3283         -  "                         The --new starts with an empty file\n"
         3364  +  ".open ?OPTIONS? ?FILE? Close existing database and reopen FILE\n"
         3365  +  "                         The --new option starts with an empty file\n"
  3284   3366     ".output ?FILENAME?     Send output to FILENAME or stdout\n"
  3285   3367     ".print STRING...       Print literal STRING\n"
  3286   3368     ".prompt MAIN CONTINUE  Replace the standard prompts\n"
  3287   3369     ".quit                  Exit this program\n"
  3288   3370     ".read FILENAME         Execute SQL in FILENAME\n"
  3289   3371     ".restore ?DB? FILE     Restore content of DB (default \"main\") from FILE\n"
  3290   3372     ".save FILE             Write in-memory database into FILE\n"
................................................................................
  3341   3423   
  3342   3424   
  3343   3425   /* Forward reference */
  3344   3426   static int process_input(ShellState *p, FILE *in);
  3345   3427   
  3346   3428   /*
  3347   3429   ** Read the content of file zName into memory obtained from sqlite3_malloc64()
  3348         -** and return a pointer to the buffer. The caller is responsible for freeing 
  3349         -** the memory. 
         3430  +** and return a pointer to the buffer. The caller is responsible for freeing
         3431  +** the memory.
  3350   3432   **
  3351   3433   ** If parameter pnByte is not NULL, (*pnByte) is set to the number of bytes
  3352   3434   ** read.
  3353   3435   **
  3354   3436   ** For convenience, a nul-terminator byte is always appended to the data read
  3355   3437   ** from the file before the buffer is returned. This byte is not included in
  3356   3438   ** the final value of (*pnByte), if applicable.
................................................................................
  3505   3587                               sha3Func, 0, 0);
  3506   3588       sqlite3_create_function(p->db, "sha3", 2, SQLITE_UTF8, 0,
  3507   3589                               sha3Func, 0, 0);
  3508   3590       sqlite3_create_function(p->db, "sha3_query", 1, SQLITE_UTF8, 0,
  3509   3591                               sha3QueryFunc, 0, 0);
  3510   3592       sqlite3_create_function(p->db, "sha3_query", 2, SQLITE_UTF8, 0,
  3511   3593                               sha3QueryFunc, 0, 0);
         3594  +    sqlite3_create_function(p->db, "shell_add_schema", 2, SQLITE_UTF8, 0,
         3595  +                            shellAddSchemaName, 0, 0);
         3596  +                            
  3512   3597     }
  3513   3598   }
  3514   3599   
  3515   3600   /*
  3516   3601   ** Do C-language style dequoting.
  3517   3602   **
  3518   3603   **    \a    -> alarm
................................................................................
  4349   4434   ** by the ".lint fkey-indexes" command. This scalar function is always
  4350   4435   ** called with four arguments - the parent table name, the parent column name,
  4351   4436   ** the child table name and the child column name.
  4352   4437   **
  4353   4438   **   fkey_collate_clause('parent-tab', 'parent-col', 'child-tab', 'child-col')
  4354   4439   **
  4355   4440   ** If either of the named tables or columns do not exist, this function
  4356         -** returns an empty string. An empty string is also returned if both tables 
         4441  +** returns an empty string. An empty string is also returned if both tables
  4357   4442   ** and columns exist but have the same default collation sequence. Or,
  4358   4443   ** if both exist but the default collation sequences are different, this
  4359   4444   ** function returns the string " COLLATE <parent-collation>", where
  4360   4445   ** <parent-collation> is the default collation sequence of the parent column.
  4361   4446   */
  4362   4447   static void shellFkeyCollateClause(
  4363         -  sqlite3_context *pCtx, 
  4364         -  int nVal, 
         4448  +  sqlite3_context *pCtx,
         4449  +  int nVal,
  4365   4450     sqlite3_value **apVal
  4366   4451   ){
  4367   4452     sqlite3 *db = sqlite3_context_db_handle(pCtx);
  4368   4453     const char *zParent;
  4369   4454     const char *zParentCol;
  4370   4455     const char *zParentSeq;
  4371   4456     const char *zChild;
  4372   4457     const char *zChildCol;
  4373   4458     const char *zChildSeq = 0;  /* Initialize to avoid false-positive warning */
  4374   4459     int rc;
  4375         -  
         4460  +
  4376   4461     assert( nVal==4 );
  4377   4462     zParent = (const char*)sqlite3_value_text(apVal[0]);
  4378   4463     zParentCol = (const char*)sqlite3_value_text(apVal[1]);
  4379   4464     zChild = (const char*)sqlite3_value_text(apVal[2]);
  4380   4465     zChildCol = (const char*)sqlite3_value_text(apVal[3]);
  4381   4466   
  4382   4467     sqlite3_result_text(pCtx, "", -1, SQLITE_STATIC);
................................................................................
  4490   4575       else{
  4491   4576         raw_printf(stderr, "Usage: %s %s ?-verbose? ?-groupbyparent?\n",
  4492   4577             azArg[0], azArg[1]
  4493   4578         );
  4494   4579         return SQLITE_ERROR;
  4495   4580       }
  4496   4581     }
  4497         -  
         4582  +
  4498   4583     /* Register the fkey_collate_clause() SQL function */
  4499   4584     rc = sqlite3_create_function(db, "fkey_collate_clause", 4, SQLITE_UTF8,
  4500   4585         0, shellFkeyCollateClause, 0, 0
  4501   4586     );
  4502   4587   
  4503   4588   
  4504   4589     if( rc==SQLITE_OK ){
................................................................................
  4533   4618         rc = sqlite3_finalize(pExplain);
  4534   4619         if( rc!=SQLITE_OK ) break;
  4535   4620   
  4536   4621         if( res<0 ){
  4537   4622           raw_printf(stderr, "Error: internal error");
  4538   4623           break;
  4539   4624         }else{
  4540         -        if( bGroupByParent 
         4625  +        if( bGroupByParent
  4541   4626           && (bVerbose || res==0)
  4542         -        && (zPrev==0 || sqlite3_stricmp(zParent, zPrev)) 
         4627  +        && (zPrev==0 || sqlite3_stricmp(zParent, zPrev))
  4543   4628           ){
  4544   4629             raw_printf(out, "-- Parent table %s\n", zParent);
  4545   4630             sqlite3_free(zPrev);
  4546   4631             zPrev = sqlite3_mprintf("%s", zParent);
  4547   4632           }
  4548   4633   
  4549   4634           if( res==0 ){
  4550   4635             raw_printf(out, "%s%s --> %s\n", zIndent, zCI, zTarget);
  4551   4636           }else if( bVerbose ){
  4552         -          raw_printf(out, "%s/* no extra indexes required for %s -> %s */\n", 
         4637  +          raw_printf(out, "%s/* no extra indexes required for %s -> %s */\n",
  4553   4638                 zIndent, zFrom, zTarget
  4554   4639             );
  4555   4640           }
  4556   4641         }
  4557   4642       }
  4558   4643       sqlite3_free(zPrev);
  4559   4644   
................................................................................
  4726   4811           setTextMode(p->out, 1);
  4727   4812         }
  4728   4813       }else{
  4729   4814         raw_printf(stderr, "Usage: .binary on|off\n");
  4730   4815         rc = 1;
  4731   4816       }
  4732   4817     }else
         4818  +
         4819  +  if( c=='c' && strcmp(azArg[0],"cd")==0 ){
         4820  +    if( nArg==2 ){
         4821  +#if defined(_WIN32) || defined(WIN32)
         4822  +      wchar_t *z = sqlite3_win32_utf8_to_unicode(azArg[1]);
         4823  +      rc = !SetCurrentDirectoryW(z);
         4824  +      sqlite3_free(z);
         4825  +#else
         4826  +      rc = chdir(azArg[1]);
         4827  +#endif
         4828  +      if( rc ){
         4829  +        utf8_printf(stderr, "Cannot change to directory \"%s\"\n", azArg[1]);
         4830  +        rc = 1;
         4831  +      }
         4832  +    }else{
         4833  +      raw_printf(stderr, "Usage: .cd DIRECTORY\n");
         4834  +      rc = 1;
         4835  +    }
         4836  +  }else
  4733   4837   
  4734   4838     /* The undocumented ".breakpoint" command causes a call to the no-op
  4735   4839     ** routine named test_breakpoint().
  4736   4840     */
  4737   4841     if( c=='b' && n>=3 && strncmp(azArg[0], "breakpoint", n)==0 ){
  4738   4842       test_breakpoint();
  4739   4843     }else
................................................................................
  5639   5743       }else{
  5640   5744         raw_printf(stderr, "Usage: .scanstats on|off\n");
  5641   5745         rc = 1;
  5642   5746       }
  5643   5747     }else
  5644   5748   
  5645   5749     if( c=='s' && strncmp(azArg[0], "schema", n)==0 ){
         5750  +    ShellText sSelect;
  5646   5751       ShellState data;
  5647   5752       char *zErrMsg = 0;
         5753  +    const char *zDiv = 0;
         5754  +    int iSchema = 0;
         5755  +
  5648   5756       open_db(p, 0);
  5649   5757       memcpy(&data, p, sizeof(data));
  5650   5758       data.showHeader = 0;
  5651   5759       data.cMode = data.mode = MODE_Semi;
         5760  +    initText(&sSelect);
  5652   5761       if( nArg>=2 && optionMatch(azArg[1], "indent") ){
  5653   5762         data.cMode = data.mode = MODE_Pretty;
  5654   5763         nArg--;
  5655   5764         if( nArg==2 ) azArg[1] = azArg[2];
  5656   5765       }
  5657   5766       if( nArg==2 && azArg[1][0]!='-' ){
  5658   5767         int i;
................................................................................
  5682   5791                         ")";
  5683   5792           new_argv[1] = 0;
  5684   5793           new_colv[0] = "sql";
  5685   5794           new_colv[1] = 0;
  5686   5795           callback(&data, 1, new_argv, new_colv);
  5687   5796           rc = SQLITE_OK;
  5688   5797         }else{
  5689         -        char *zSql;
  5690         -        zSql = sqlite3_mprintf(
  5691         -          "SELECT sql FROM "
  5692         -          "  (SELECT sql sql, type type, tbl_name tbl_name, name name, rowid x"
  5693         -          "     FROM sqlite_master UNION ALL"
  5694         -          "   SELECT sql, type, tbl_name, name, rowid FROM sqlite_temp_master) "
  5695         -          "WHERE lower(tbl_name) LIKE %Q"
  5696         -          "  AND type!='meta' AND sql NOTNULL "
  5697         -          "ORDER BY rowid", azArg[1]);
  5698         -        rc = sqlite3_exec(p->db, zSql, callback, &data, &zErrMsg);
  5699         -        sqlite3_free(zSql);
         5798  +        zDiv = "(";
  5700   5799         }
  5701   5800       }else if( nArg==1 ){
  5702         -      rc = sqlite3_exec(p->db,
  5703         -         "SELECT sql FROM "
  5704         -         "  (SELECT sql sql, type type, tbl_name tbl_name, name name, rowid x"
  5705         -         "     FROM sqlite_master UNION ALL"
  5706         -         "   SELECT sql, type, tbl_name, name, rowid FROM sqlite_temp_master) "
  5707         -         "WHERE type!='meta' AND sql NOTNULL AND name NOT LIKE 'sqlite_%' "
  5708         -         "ORDER BY rowid",
  5709         -         callback, &data, &zErrMsg
  5710         -      );
         5801  +      zDiv = "(";
  5711   5802       }else{
  5712   5803         raw_printf(stderr, "Usage: .schema ?--indent? ?LIKE-PATTERN?\n");
  5713   5804         rc = 1;
  5714   5805         goto meta_command_exit;
  5715   5806       }
         5807  +    if( zDiv ){
         5808  +      sqlite3_stmt *pStmt = 0;
         5809  +      sqlite3_prepare_v2(p->db, "SELECT name FROM pragma_database_list",
         5810  +                         -1, &pStmt, 0);
         5811  +      appendText(&sSelect, "SELECT sql FROM", 0);
         5812  +      iSchema = 0;
         5813  +      while( sqlite3_step(pStmt)==SQLITE_ROW ){
         5814  +        const char *zDb = (const char*)sqlite3_column_text(pStmt, 0);
         5815  +        char zScNum[30];
         5816  +        sqlite3_snprintf(sizeof(zScNum), zScNum, "%d", ++iSchema);
         5817  +        appendText(&sSelect, zDiv, 0);
         5818  +        zDiv = " UNION ALL ";
         5819  +        if( strcmp(zDb, "main")!=0 ){
         5820  +          appendText(&sSelect, "SELECT shell_add_schema(sql,", 0);
         5821  +          appendText(&sSelect, zDb, '"');
         5822  +          appendText(&sSelect, ") AS sql, type, tbl_name, name, rowid,", 0);
         5823  +          appendText(&sSelect, zScNum, 0);
         5824  +          appendText(&sSelect, " AS snum, ", 0);
         5825  +          appendText(&sSelect, zDb, '\'');
         5826  +          appendText(&sSelect, " AS sname FROM ", 0);
         5827  +          appendText(&sSelect, zDb, '"');
         5828  +          appendText(&sSelect, ".sqlite_master", 0);
         5829  +        }else{
         5830  +          appendText(&sSelect, "SELECT sql, type, tbl_name, name, rowid, ", 0);
         5831  +          appendText(&sSelect, zScNum, 0);
         5832  +          appendText(&sSelect, " AS snum, 'main' AS sname FROM sqlite_master",0);
         5833  +        }
         5834  +      }
         5835  +      sqlite3_finalize(pStmt);
         5836  +      appendText(&sSelect, ") WHERE ", 0);
         5837  +      if( nArg>1 ){
         5838  +        char *zQarg = sqlite3_mprintf("%Q", azArg[1]);
         5839  +        if( strchr(azArg[1], '.') ){
         5840  +          appendText(&sSelect, "lower(printf('%s.%s',sname,tbl_name))", 0);
         5841  +        }else{
         5842  +          appendText(&sSelect, "lower(tbl_name)", 0);
         5843  +        }
         5844  +        appendText(&sSelect, strchr(azArg[1], '*') ? " GLOB " : " LIKE ", 0);
         5845  +        appendText(&sSelect, zQarg, 0);
         5846  +        appendText(&sSelect, " AND ", 0);
         5847  +        sqlite3_free(zQarg);
         5848  +      }
         5849  +      appendText(&sSelect, "type!='meta' AND sql IS NOT NULL"
         5850  +                           " ORDER BY snum, rowid", 0);
         5851  +      rc = sqlite3_exec(p->db, sSelect.z, callback, &data, &zErrMsg);
         5852  +      freeText(&sSelect);
         5853  +    }
  5716   5854       if( zErrMsg ){
  5717   5855         utf8_printf(stderr,"Error: %s\n", zErrMsg);
  5718   5856         sqlite3_free(zErrMsg);
  5719   5857         rc = 1;
  5720   5858       }else if( rc != SQLITE_OK ){
  5721   5859         raw_printf(stderr,"Error: querying schema information\n");
  5722   5860         rc = 1;
................................................................................
  5950   6088     }else
  5951   6089   #endif
  5952   6090   
  5953   6091     if( c=='s' && n>=4 && strncmp(azArg[0],"selftest",n)==0 ){
  5954   6092       int bIsInit = 0;         /* True to initialize the SELFTEST table */
  5955   6093       int bVerbose = 0;        /* Verbose output */
  5956   6094       int bSelftestExists;     /* True if SELFTEST already exists */
  5957         -    char **azTest = 0;       /* Content of the SELFTEST table */
  5958         -    int nRow = 0;            /* Number of rows in the SELFTEST table */
  5959         -    int nCol = 4;            /* Number of columns in the SELFTEST table */
  5960         -    int i;                   /* Loop counter */
         6095  +    int i, k;                /* Loop counters */
  5961   6096       int nTest = 0;           /* Number of tests runs */
  5962   6097       int nErr = 0;            /* Number of errors seen */
  5963   6098       ShellText str;           /* Answer for a query */
  5964         -    static char *azDefaultTest[] = {
  5965         -       0, 0, 0, 0,
  5966         -       "0", "memo", "Missing SELFTEST table - default checks only", "",
  5967         -       "1", "run", "PRAGMA integrity_check", "ok"
  5968         -    };
  5969         -    static const int nDefaultRow = 2;
         6099  +    sqlite3_stmt *pStmt = 0; /* Query against the SELFTEST table */
  5970   6100   
  5971   6101       open_db(p,0);
  5972   6102       for(i=1; i<nArg; i++){
  5973   6103         const char *z = azArg[i];
  5974   6104         if( z[0]=='-' && z[1]=='-' ) z++;
  5975   6105         if( strcmp(z,"-init")==0 ){
  5976   6106           bIsInit = 1;
................................................................................
  5992   6122       }else{
  5993   6123         bSelftestExists = 1;
  5994   6124       }
  5995   6125       if( bIsInit ){
  5996   6126         createSelftestTable(p);
  5997   6127         bSelftestExists = 1;
  5998   6128       }
  5999         -    if( bSelftestExists ){
  6000         -      rc = sqlite3_get_table(p->db, 
  6001         -          "SELECT tno,op,cmd,ans FROM selftest ORDER BY tno",
  6002         -          &azTest, &nRow, &nCol, 0);
         6129  +    initText(&str);
         6130  +    appendText(&str, "x", 0);
         6131  +    for(k=bSelftestExists; k>=0; k--){
         6132  +      if( k==1 ){
         6133  +        rc = sqlite3_prepare_v2(p->db,
         6134  +            "SELECT tno,op,cmd,ans FROM selftest ORDER BY tno",
         6135  +            -1, &pStmt, 0);
         6136  +      }else{
         6137  +        rc = sqlite3_prepare_v2(p->db,
         6138  +          "VALUES(0,'memo','Missing SELFTEST table - default checks only',''),"
         6139  +          "      (1,'run','PRAGMA integrity_check','ok')",
         6140  +          -1, &pStmt, 0);
         6141  +      }
  6003   6142         if( rc ){
  6004   6143           raw_printf(stderr, "Error querying the selftest table\n");
  6005   6144           rc = 1;
  6006         -        sqlite3_free_table(azTest);
         6145  +        sqlite3_finalize(pStmt);
  6007   6146           goto meta_command_exit;
  6008         -      }else if( nRow==0 ){
  6009         -        sqlite3_free_table(azTest);
  6010         -        azTest = azDefaultTest;
  6011         -        nRow = nDefaultRow;
  6012   6147         }
  6013         -    }else{
  6014         -      azTest = azDefaultTest;
  6015         -      nRow = nDefaultRow;
  6016         -    }
  6017         -    initText(&str);
  6018         -    appendText(&str, "x", 0);
  6019         -    for(i=1; i<=nRow; i++){
  6020         -      int tno = atoi(azTest[i*nCol]);
  6021         -      const char *zOp = azTest[i*nCol+1];
  6022         -      const char *zSql = azTest[i*nCol+2];
  6023         -      const char *zAns = azTest[i*nCol+3];
  6024         -  
  6025         -      if( bVerbose>0 ){
  6026         -        char *zQuote = sqlite3_mprintf("%q", zSql);
  6027         -        printf("%d: %s %s\n", tno, zOp, zSql);
  6028         -        sqlite3_free(zQuote);
  6029         -      }
  6030         -      if( strcmp(zOp,"memo")==0 ){
  6031         -        utf8_printf(p->out, "%s\n", zSql);
  6032         -      }else
  6033         -      if( strcmp(zOp,"run")==0 ){
  6034         -        char *zErrMsg = 0;
  6035         -        str.n = 0;
  6036         -        str.z[0] = 0;
  6037         -        rc = sqlite3_exec(p->db, zSql, captureOutputCallback, &str, &zErrMsg);
  6038         -        nTest++;
  6039         -        if( bVerbose ){
  6040         -          utf8_printf(p->out, "Result: %s\n", str.z);
         6148  +      for(i=1; sqlite3_step(pStmt)==SQLITE_ROW; i++){
         6149  +        int tno = sqlite3_column_int(pStmt, 0);
         6150  +        const char *zOp = (const char*)sqlite3_column_text(pStmt, 1);
         6151  +        const char *zSql = (const char*)sqlite3_column_text(pStmt, 2);
         6152  +        const char *zAns = (const char*)sqlite3_column_text(pStmt, 3);
         6153  +
         6154  +        k = 0;
         6155  +        if( bVerbose>0 ){
         6156  +          char *zQuote = sqlite3_mprintf("%q", zSql);
         6157  +          printf("%d: %s %s\n", tno, zOp, zSql);
         6158  +          sqlite3_free(zQuote);
  6041   6159           }
  6042         -        if( rc || zErrMsg ){
  6043         -          nErr++;
         6160  +        if( strcmp(zOp,"memo")==0 ){
         6161  +          utf8_printf(p->out, "%s\n", zSql);
         6162  +        }else
         6163  +        if( strcmp(zOp,"run")==0 ){
         6164  +          char *zErrMsg = 0;
         6165  +          str.n = 0;
         6166  +          str.z[0] = 0;
         6167  +          rc = sqlite3_exec(p->db, zSql, captureOutputCallback, &str, &zErrMsg);
         6168  +          nTest++;
         6169  +          if( bVerbose ){
         6170  +            utf8_printf(p->out, "Result: %s\n", str.z);
         6171  +          }
         6172  +          if( rc || zErrMsg ){
         6173  +            nErr++;
         6174  +            rc = 1;
         6175  +            utf8_printf(p->out, "%d: error-code-%d: %s\n", tno, rc, zErrMsg);
         6176  +            sqlite3_free(zErrMsg);
         6177  +          }else if( strcmp(zAns,str.z)!=0 ){
         6178  +            nErr++;
         6179  +            rc = 1;
         6180  +            utf8_printf(p->out, "%d: Expected: [%s]\n", tno, zAns);
         6181  +            utf8_printf(p->out, "%d:      Got: [%s]\n", tno, str.z);
         6182  +          }
         6183  +        }else
         6184  +        {
         6185  +          utf8_printf(stderr,
         6186  +            "Unknown operation \"%s\" on selftest line %d\n", zOp, tno);
  6044   6187             rc = 1;
  6045         -          utf8_printf(p->out, "%d: error-code-%d: %s\n", tno, rc, zErrMsg);
  6046         -          sqlite3_free(zErrMsg);
  6047         -        }else if( strcmp(zAns,str.z)!=0 ){
  6048         -          nErr++;
  6049         -          rc = 1;
  6050         -          utf8_printf(p->out, "%d: Expected: [%s]\n", tno, zAns);
  6051         -          utf8_printf(p->out, "%d:      Got: [%s]\n", tno, str.z);
         6188  +          break;
  6052   6189           }
  6053         -      }else
  6054         -      {
  6055         -        utf8_printf(stderr,
  6056         -          "Unknown operation \"%s\" on selftest line %d\n", zOp, tno);
  6057         -        rc = 1;
  6058         -        break;
  6059         -      }
  6060         -    }
         6190  +      } /* End loop over rows of content from SELFTEST */
         6191  +      sqlite3_finalize(pStmt);
         6192  +    } /* End loop over k */
  6061   6193       freeText(&str);
  6062         -    if( azTest!=azDefaultTest ) sqlite3_free_table(azTest);
  6063   6194       utf8_printf(p->out, "%d errors out of %d tests\n", nErr, nTest);
  6064   6195     }else
  6065   6196   
  6066   6197     if( c=='s' && strncmp(azArg[0], "separator", n)==0 ){
  6067   6198       if( nArg<2 || nArg>3 ){
  6068   6199         raw_printf(stderr, "Usage: .separator COL ?ROW?\n");
  6069   6200         rc = 1;
................................................................................
  6095   6226         const char *z = azArg[i];
  6096   6227         if( z[0]=='-' ){
  6097   6228           z++;
  6098   6229           if( z[0]=='-' ) z++;
  6099   6230           if( strcmp(z,"schema")==0 ){
  6100   6231             bSchema = 1;
  6101   6232           }else
  6102         -        if( strcmp(z,"sha3-224")==0 || strcmp(z,"sha3-256")==0 
  6103         -         || strcmp(z,"sha3-384")==0 || strcmp(z,"sha3-512")==0 
         6233  +        if( strcmp(z,"sha3-224")==0 || strcmp(z,"sha3-256")==0
         6234  +         || strcmp(z,"sha3-384")==0 || strcmp(z,"sha3-512")==0
  6104   6235           ){
  6105   6236             iSize = atoi(&z[5]);
  6106   6237           }else
  6107   6238           if( strcmp(z,"debug")==0 ){
  6108   6239             bDebug = 1;
  6109   6240           }else
  6110   6241           {
................................................................................
  6264   6395     if( (c=='t' && n>1 && strncmp(azArg[0], "tables", n)==0)
  6265   6396      || (c=='i' && (strncmp(azArg[0], "indices", n)==0
  6266   6397                    || strncmp(azArg[0], "indexes", n)==0) )
  6267   6398     ){
  6268   6399       sqlite3_stmt *pStmt;
  6269   6400       char **azResult;
  6270   6401       int nRow, nAlloc;
  6271         -    char *zSql = 0;
  6272   6402       int ii;
         6403  +    ShellText s;
         6404  +    initText(&s);
  6273   6405       open_db(p, 0);
  6274   6406       rc = sqlite3_prepare_v2(p->db, "PRAGMA database_list", -1, &pStmt, 0);
  6275   6407       if( rc ) return shellDatabaseError(p->db);
  6276   6408   
  6277         -    /* Create an SQL statement to query for the list of tables in the
  6278         -    ** main and all attached databases where the table name matches the
  6279         -    ** LIKE pattern bound to variable "?1". */
  6280         -    if( c=='t' ){
  6281         -      zSql = sqlite3_mprintf(
  6282         -          "SELECT name FROM sqlite_master"
  6283         -          " WHERE type IN ('table','view')"
  6284         -          "   AND name NOT LIKE 'sqlite_%%'"
  6285         -          "   AND name LIKE ?1");
  6286         -    }else if( nArg>2 ){
         6409  +    if( nArg>2 && c=='i' ){
  6287   6410         /* It is an historical accident that the .indexes command shows an error
  6288   6411         ** when called with the wrong number of arguments whereas the .tables
  6289   6412         ** command does not. */
  6290   6413         raw_printf(stderr, "Usage: .indexes ?LIKE-PATTERN?\n");
  6291   6414         rc = 1;
  6292   6415         goto meta_command_exit;
  6293         -    }else{
  6294         -      zSql = sqlite3_mprintf(
  6295         -          "SELECT name FROM sqlite_master"
  6296         -          " WHERE type='index'"
  6297         -          "   AND tbl_name LIKE ?1");
  6298   6416       }
  6299         -    for(ii=0; zSql && sqlite3_step(pStmt)==SQLITE_ROW; ii++){
         6417  +    for(ii=0; sqlite3_step(pStmt)==SQLITE_ROW; ii++){
  6300   6418         const char *zDbName = (const char*)sqlite3_column_text(pStmt, 1);
  6301         -      if( zDbName==0 || ii==0 ) continue;
         6419  +      if( zDbName==0 ) continue;
         6420  +      if( s.z && s.z[0] ) appendText(&s, " UNION ALL ", 0);
         6421  +      if( sqlite3_stricmp(zDbName, "main")==0 ){
         6422  +        appendText(&s, "SELECT name FROM ", 0);
         6423  +      }else{
         6424  +        appendText(&s, "SELECT ", 0);
         6425  +        appendText(&s, zDbName, '\'');
         6426  +        appendText(&s, "||'.'||name FROM ", 0);
         6427  +      }
         6428  +      appendText(&s, zDbName, '"');
         6429  +      appendText(&s, ".sqlite_master ", 0);
  6302   6430         if( c=='t' ){
  6303         -        zSql = sqlite3_mprintf(
  6304         -                 "%z UNION ALL "
  6305         -                 "SELECT '%q.' || name FROM \"%w\".sqlite_master"
  6306         -                 " WHERE type IN ('table','view')"
  6307         -                 "   AND name NOT LIKE 'sqlite_%%'"
  6308         -                 "   AND name LIKE ?1", zSql, zDbName, zDbName);
         6431  +        appendText(&s," WHERE type IN ('table','view')"
         6432  +                      "   AND name NOT LIKE 'sqlite_%'"
         6433  +                      "   AND name LIKE ?1", 0);
  6309   6434         }else{
  6310         -        zSql = sqlite3_mprintf(
  6311         -                 "%z UNION ALL "
  6312         -                 "SELECT '%q.' || name FROM \"%w\".sqlite_master"
  6313         -                 " WHERE type='index'"
  6314         -                 "   AND tbl_name LIKE ?1", zSql, zDbName, zDbName);
         6435  +        appendText(&s," WHERE type='index'"
         6436  +                      "   AND tbl_name LIKE ?1", 0);
  6315   6437         }
  6316   6438       }
  6317   6439       rc = sqlite3_finalize(pStmt);
  6318         -    if( zSql && rc==SQLITE_OK ){
  6319         -      zSql = sqlite3_mprintf("%z ORDER BY 1", zSql);
  6320         -      if( zSql ) rc = sqlite3_prepare_v2(p->db, zSql, -1, &pStmt, 0);
  6321         -    }
  6322         -    sqlite3_free(zSql);
  6323         -    if( !zSql ) return shellNomemError();
         6440  +    appendText(&s, " ORDER BY 1", 0);
         6441  +    rc = sqlite3_prepare_v2(p->db, s.z, -1, &pStmt, 0);
         6442  +    freeText(&s);
  6324   6443       if( rc ) return shellDatabaseError(p->db);
  6325   6444   
  6326   6445       /* Run the SQL statement prepared by the above block. Store the results
  6327   6446       ** as an array of nul-terminated strings in azResult[].  */
  6328   6447       nRow = nAlloc = 0;
  6329   6448       azResult = 0;
  6330   6449       if( nArg>1 ){

Changes to src/sqlite.h.in.

  5617   5617   ** interface returns SQLITE_OK and fills in the non-NULL pointers in
  5618   5618   ** the final five arguments with appropriate values if the specified
  5619   5619   ** column exists.  ^The sqlite3_table_column_metadata() interface returns
  5620   5620   ** SQLITE_ERROR and if the specified column does not exist.
  5621   5621   ** ^If the column-name parameter to sqlite3_table_column_metadata() is a
  5622   5622   ** NULL pointer, then this routine simply checks for the existence of the
  5623   5623   ** table and returns SQLITE_OK if the table exists and SQLITE_ERROR if it
  5624         -** does not.
         5624  +** does not.  If the table name parameter T in a call to
         5625  +** sqlite3_table_column_metadata(X,D,T,C,...) is NULL then the result is
         5626  +** undefined behavior.
  5625   5627   **
  5626   5628   ** ^The column is identified by the second, third and fourth parameters to
  5627   5629   ** this function. ^(The second parameter is either the name of the database
  5628   5630   ** (i.e. "main", "temp", or an attached database) containing the specified
  5629   5631   ** table or NULL.)^ ^If it is NULL, then all attached databases are searched
  5630   5632   ** for the table using the same algorithm used by the database engine to
  5631   5633   ** resolve unqualified table references.
................................................................................
  7130   7132   ** [[SQLITE_STMTSTATUS_VM_STEP]] <dt>SQLITE_STMTSTATUS_VM_STEP</dt>
  7131   7133   ** <dd>^This is the number of virtual machine operations executed
  7132   7134   ** by the prepared statement if that number is less than or equal
  7133   7135   ** to 2147483647.  The number of virtual machine operations can be 
  7134   7136   ** used as a proxy for the total work done by the prepared statement.
  7135   7137   ** If the number of virtual machine operations exceeds 2147483647
  7136   7138   ** then the value returned by this statement status code is undefined.
         7139  +**
         7140  +** [[SQLITE_STMTSTATUS_MEMUSED]] <dt>SQLITE_STMTSTATUS_MEMUSED</dt>
         7141  +** <dd>^This is the approximate number of bytes of heap memory
         7142  +** used to store the prepared statement.  ^This value is not actually
         7143  +** a counter, and so the resetFlg parameter to sqlite3_stmt_status()
         7144  +** is ignored when the opcode is SQLITE_STMTSTATUS_MEMUSED.
  7137   7145   ** </dd>
  7138   7146   ** </dl>
  7139   7147   */
  7140   7148   #define SQLITE_STMTSTATUS_FULLSCAN_STEP     1
  7141   7149   #define SQLITE_STMTSTATUS_SORT              2
  7142   7150   #define SQLITE_STMTSTATUS_AUTOINDEX         3
  7143   7151   #define SQLITE_STMTSTATUS_VM_STEP           4
         7152  +#define SQLITE_STMTSTATUS_MEMUSED           5
  7144   7153   
  7145   7154   /*
  7146   7155   ** CAPI3REF: Custom Page Cache Object
  7147   7156   **
  7148   7157   ** The sqlite3_pcache type is opaque.  It is implemented by
  7149   7158   ** the pluggable module.  The SQLite core has no knowledge of
  7150   7159   ** its size or internal structure and never deals with the

Changes to src/sqliteInt.h.

   270    270   ** threadsafe.  1 means the library is serialized which is the highest
   271    271   ** level of threadsafety.  2 means the library is multithreaded - multiple
   272    272   ** threads can use SQLite as long as no two threads try to use the same
   273    273   ** database connection at the same time.
   274    274   **
   275    275   ** Older versions of SQLite used an optional THREADSAFE macro.
   276    276   ** We support that for legacy.
          277  +**
          278  +** To ensure that the correct value of "THREADSAFE" is reported when querying
          279  +** for compile-time options at runtime (e.g. "PRAGMA compile_options"), this
          280  +** logic is partially replicated in ctime.c. If it is updated here, it should
          281  +** also be updated there.
   277    282   */
   278    283   #if !defined(SQLITE_THREADSAFE)
   279    284   # if defined(THREADSAFE)
   280    285   #   define SQLITE_THREADSAFE THREADSAFE
   281    286   # else
   282    287   #   define SQLITE_THREADSAFE 1 /* IMP: R-07272-22309 */
   283    288   # endif
................................................................................
   585    590   
   586    591   /*
   587    592   ** Provide a default value for SQLITE_TEMP_STORE in case it is not specified
   588    593   ** on the command-line
   589    594   */
   590    595   #ifndef SQLITE_TEMP_STORE
   591    596   # define SQLITE_TEMP_STORE 1
   592         -# define SQLITE_TEMP_STORE_xc 1  /* Exclude from ctime.c */
   593    597   #endif
   594    598   
   595    599   /*
   596    600   ** If no value has been provided for SQLITE_MAX_WORKER_THREADS, or if
   597    601   ** SQLITE_TEMP_STORE is set to 3 (never use temporary files), set it
   598    602   ** to zero.
   599    603   */
................................................................................
   886    890     || defined(__sun) \
   887    891     || defined(__FreeBSD__) \
   888    892     || defined(__DragonFly__)
   889    893   #   define SQLITE_MAX_MMAP_SIZE 0x7fff0000  /* 2147418112 */
   890    894   # else
   891    895   #   define SQLITE_MAX_MMAP_SIZE 0
   892    896   # endif
   893         -# define SQLITE_MAX_MMAP_SIZE_xc 1 /* exclude from ctime.c */
   894    897   #endif
   895    898   
   896    899   /*
   897    900   ** The default MMAP_SIZE is zero on all platforms.  Or, even if a larger
   898    901   ** default MMAP_SIZE is specified at compile-time, make sure that it does
   899    902   ** not exceed the maximum mmap size.
   900    903   */
   901    904   #ifndef SQLITE_DEFAULT_MMAP_SIZE
   902    905   # define SQLITE_DEFAULT_MMAP_SIZE 0
   903         -# define SQLITE_DEFAULT_MMAP_SIZE_xc 1  /* Exclude from ctime.c */
   904    906   #endif
   905    907   #if SQLITE_DEFAULT_MMAP_SIZE>SQLITE_MAX_MMAP_SIZE
   906    908   # undef SQLITE_DEFAULT_MMAP_SIZE
   907    909   # define SQLITE_DEFAULT_MMAP_SIZE SQLITE_MAX_MMAP_SIZE
   908    910   #endif
   909    911   
   910    912   /*
................................................................................
  2360   2362   
  2361   2363   /*
  2362   2364   ** The following are the meanings of bits in the Expr.flags field.
  2363   2365   */
  2364   2366   #define EP_FromJoin  0x000001 /* Originates in ON/USING clause of outer join */
  2365   2367   #define EP_Agg       0x000002 /* Contains one or more aggregate functions */
  2366   2368   #define EP_Resolved  0x000004 /* IDs have been resolved to COLUMNs */
  2367         -#define EP_Error     0x000008 /* Expression contains one or more errors */
         2369  +                  /* 0x000008 // available for use */
  2368   2370   #define EP_Distinct  0x000010 /* Aggregate function with DISTINCT keyword */
  2369   2371   #define EP_VarSelect 0x000020 /* pSelect is correlated, not constant */
  2370   2372   #define EP_DblQuoted 0x000040 /* token.z was originally in "..." */
  2371   2373   #define EP_InfixFunc 0x000080 /* True for an infix function: LIKE, GLOB, etc */
  2372   2374   #define EP_Collate   0x000100 /* Tree contains a TK_COLLATE operator */
  2373   2375   #define EP_Generic   0x000200 /* Ignore COLLATE or affinity on this tree */
  2374   2376   #define EP_IntValue  0x000400 /* Integer value contained in u.iValue */
................................................................................
  2827   2829   
  2828   2830   /*
  2829   2831   ** An instance of this object describes where to put of the results of
  2830   2832   ** a SELECT statement.
  2831   2833   */
  2832   2834   struct SelectDest {
  2833   2835     u8 eDest;            /* How to dispose of the results.  On of SRT_* above. */
  2834         -  char *zAffSdst;      /* Affinity used when eDest==SRT_Set */
  2835   2836     int iSDParm;         /* A parameter used by the eDest disposal method */
  2836   2837     int iSdst;           /* Base register where results are written */
  2837   2838     int nSdst;           /* Number of registers allocated */
         2839  +  char *zAffSdst;      /* Affinity used when eDest==SRT_Set */
  2838   2840     ExprList *pOrderBy;  /* Key columns for SRT_Queue and SRT_DistQueue */
  2839   2841   };
  2840   2842   
  2841   2843   /*
  2842   2844   ** During code generation of statements that do inserts into AUTOINCREMENT
  2843   2845   ** tables, the following information is attached to the Table.u.autoInc.p
  2844   2846   ** pointer of each autoincrement table to record some side information that
................................................................................
  3333   3335   /* Forward declarations */
  3334   3336   int sqlite3WalkExpr(Walker*, Expr*);
  3335   3337   int sqlite3WalkExprList(Walker*, ExprList*);
  3336   3338   int sqlite3WalkSelect(Walker*, Select*);
  3337   3339   int sqlite3WalkSelectExpr(Walker*, Select*);
  3338   3340   int sqlite3WalkSelectFrom(Walker*, Select*);
  3339   3341   int sqlite3ExprWalkNoop(Walker*, Expr*);
         3342  +int sqlite3SelectWalkNoop(Walker*, Select*);
         3343  +#ifdef SQLITE_DEBUG
         3344  +void sqlite3SelectWalkAssert2(Walker*, Select*);
         3345  +#endif
  3340   3346   
  3341   3347   /*
  3342   3348   ** Return code from the parse-tree walking primitives and their
  3343   3349   ** callbacks.
  3344   3350   */
  3345   3351   #define WRC_Continue    0   /* Continue down into children */
  3346   3352   #define WRC_Prune       1   /* Omit children but continue walking siblings */
................................................................................
  3394   3400   int sqlite3CantopenError(int);
  3395   3401   #define SQLITE_CORRUPT_BKPT sqlite3CorruptError(__LINE__)
  3396   3402   #define SQLITE_MISUSE_BKPT sqlite3MisuseError(__LINE__)
  3397   3403   #define SQLITE_CANTOPEN_BKPT sqlite3CantopenError(__LINE__)
  3398   3404   #ifdef SQLITE_DEBUG
  3399   3405     int sqlite3NomemError(int);
  3400   3406     int sqlite3IoerrnomemError(int);
         3407  +  int sqlite3CorruptPgnoError(int,Pgno);
  3401   3408   # define SQLITE_NOMEM_BKPT sqlite3NomemError(__LINE__)
  3402   3409   # define SQLITE_IOERR_NOMEM_BKPT sqlite3IoerrnomemError(__LINE__)
         3410  +# define SQLITE_CORRUPT_PGNO(P) sqlite3CorruptPgnoError(__LINE__,(P))
  3403   3411   #else
  3404   3412   # define SQLITE_NOMEM_BKPT SQLITE_NOMEM
  3405   3413   # define SQLITE_IOERR_NOMEM_BKPT SQLITE_IOERR_NOMEM
         3414  +# define SQLITE_CORRUPT_PGNO(P) sqlite3CorruptError(__LINE__)
  3406   3415   #endif
  3407   3416   
  3408   3417   /*
  3409   3418   ** FTS3 and FTS4 both require virtual table support
  3410   3419   */
  3411   3420   #if defined(SQLITE_OMIT_VIRTUALTABLE)
  3412   3421   # undef SQLITE_ENABLE_FTS3
................................................................................
  4382   4391   #endif
  4383   4392   
  4384   4393   int sqlite3ExprVectorSize(Expr *pExpr);
  4385   4394   int sqlite3ExprIsVector(Expr *pExpr);
  4386   4395   Expr *sqlite3VectorFieldSubexpr(Expr*, int);
  4387   4396   Expr *sqlite3ExprForVectorField(Parse*,Expr*,int);
  4388   4397   void sqlite3VectorErrorMsg(Parse*, Expr*);
         4398  +
         4399  +const char **sqlite3CompileOptions(int *pnOpt);
  4389   4400   
  4390   4401   #endif /* SQLITEINT_H */

Changes to src/tclsqlite.c.

   157    157     Tcl_Obj *pCollateNeeded;   /* Collation needed script */
   158    158     SqlPreparedStmt *stmtList; /* List of prepared statements*/
   159    159     SqlPreparedStmt *stmtLast; /* Last statement in the list */
   160    160     int maxStmt;               /* The next maximum number of stmtList */
   161    161     int nStmt;                 /* Number of statements in stmtList */
   162    162     IncrblobChannel *pIncrblob;/* Linked list of open incrblob channels */
   163    163     int nStep, nSort, nIndex;  /* Statistics for most recent operation */
          164  +  int nVMStep;               /* Another statistic for most recent operation */
   164    165     int nTransaction;          /* Number of nested [transaction] methods */
   165    166     int openFlags;             /* Flags used to open.  (SQLITE_OPEN_URI) */
   166    167   #ifdef SQLITE_TEST
   167    168     int bLegacyPrepare;        /* True to use sqlite3_prepare() */
   168    169   #endif
   169    170   };
   170    171   
................................................................................
  1584   1585           dbEvalRowInfo(p, 0, 0);
  1585   1586         }
  1586   1587         rcs = sqlite3_reset(pStmt);
  1587   1588   
  1588   1589         pDb->nStep = sqlite3_stmt_status(pStmt,SQLITE_STMTSTATUS_FULLSCAN_STEP,1);
  1589   1590         pDb->nSort = sqlite3_stmt_status(pStmt,SQLITE_STMTSTATUS_SORT,1);
  1590   1591         pDb->nIndex = sqlite3_stmt_status(pStmt,SQLITE_STMTSTATUS_AUTOINDEX,1);
         1592  +      pDb->nVMStep = sqlite3_stmt_status(pStmt,SQLITE_STMTSTATUS_VM_STEP,1);
  1591   1593         dbReleaseColumnNames(p);
  1592   1594         p->pPreStmt = 0;
  1593   1595   
  1594   1596         if( rcs!=SQLITE_OK ){
  1595   1597           /* If a run-time error occurs, report the error and stop reading
  1596   1598           ** the SQL.  */
  1597   1599           dbReleaseStmt(pDb, pPreStmt, 1);
................................................................................
  2851   2853         rc = TCL_ERROR;
  2852   2854       }
  2853   2855       sqlite3_close(pSrc);
  2854   2856       break;
  2855   2857     }
  2856   2858   
  2857   2859     /*
  2858         -  **     $db status (step|sort|autoindex)
         2860  +  **     $db status (step|sort|autoindex|vmstep)
  2859   2861     **
  2860   2862     ** Display SQLITE_STMTSTATUS_FULLSCAN_STEP or
  2861   2863     ** SQLITE_STMTSTATUS_SORT for the most recent eval.
  2862   2864     */
  2863   2865     case DB_STATUS: {
  2864   2866       int v;
  2865   2867       const char *zOp;
................................................................................
  2870   2872       zOp = Tcl_GetString(objv[2]);
  2871   2873       if( strcmp(zOp, "step")==0 ){
  2872   2874         v = pDb->nStep;
  2873   2875       }else if( strcmp(zOp, "sort")==0 ){
  2874   2876         v = pDb->nSort;
  2875   2877       }else if( strcmp(zOp, "autoindex")==0 ){
  2876   2878         v = pDb->nIndex;
         2879  +    }else if( strcmp(zOp, "vmstep")==0 ){
         2880  +      v = pDb->nVMStep;
  2877   2881       }else{
  2878   2882         Tcl_AppendResult(interp,
  2879         -            "bad argument: should be autoindex, step, or sort",
         2883  +            "bad argument: should be autoindex, step, sort or vmstep",
  2880   2884               (char*)0);
  2881   2885         return TCL_ERROR;
  2882   2886       }
  2883   2887       Tcl_SetObjResult(interp, Tcl_NewIntObj(v));
  2884   2888       break;
  2885   2889     }
  2886   2890   

Changes to src/test3.c.

   302    302     if( argc!=2 ){
   303    303       Tcl_AppendResult(interp, "wrong # args: should be \"", argv[0],
   304    304          " ID\"", 0);
   305    305       return TCL_ERROR;
   306    306     }
   307    307     pCur = sqlite3TestTextToPtr(argv[1]);
   308    308     sqlite3BtreeEnter(pCur->pBtree);
   309         -  rc = sqlite3BtreeNext(pCur, &res);
          309  +  rc = sqlite3BtreeNext(pCur, 0);
          310  +  if( rc==SQLITE_DONE ){
          311  +    res = 1;
          312  +    rc = SQLITE_OK;
          313  +  }
   310    314     sqlite3BtreeLeave(pCur->pBtree);
   311    315     if( rc ){
   312    316       Tcl_AppendResult(interp, sqlite3ErrName(rc), 0);
   313    317       return TCL_ERROR;
   314    318     }
   315    319     sqlite3_snprintf(sizeof(zBuf),zBuf,"%d",res);
   316    320     Tcl_AppendResult(interp, zBuf, 0);

Changes to src/test_fs.c.

   541    541         for(i=nPrefix; zQuery[i]; i++){
   542    542           if( zQuery[i]==aWild[0] || zQuery[i]==aWild[1] ) break;
   543    543           if( zQuery[i]=='/' ) nDir = i;
   544    544         }
   545    545         zDir = zQuery;
   546    546       }
   547    547     }
          548  +  if( nDir==0 ) nDir = 1;
   548    549   
   549    550     sqlite3_bind_text(pCsr->pStmt, 1, zDir, nDir, SQLITE_TRANSIENT);
   550    551     sqlite3_bind_text(pCsr->pStmt, 2, zRoot, nRoot, SQLITE_TRANSIENT);
   551    552     sqlite3_bind_text(pCsr->pStmt, 3, zPrefix, nPrefix, SQLITE_TRANSIENT);
   552    553   
   553    554   #if SQLITE_OS_WIN
   554    555     sqlite3_free(zPrefix);

Changes to src/trigger.c.

   302    302       char *z;
   303    303   
   304    304       /* Make an entry in the sqlite_master table */
   305    305       v = sqlite3GetVdbe(pParse);
   306    306       if( v==0 ) goto triggerfinish_cleanup;
   307    307       sqlite3BeginWriteOperation(pParse, 0, iDb);
   308    308       z = sqlite3DbStrNDup(db, (char*)pAll->z, pAll->n);
          309  +    testcase( z==0 );
   309    310       sqlite3NestedParse(pParse,
   310    311          "INSERT INTO %Q.%s VALUES('trigger',%Q,%Q,0,'CREATE TRIGGER %q')",
   311    312          db->aDb[iDb].zDbSName, MASTER_NAME, zName,
   312    313          pTrig->table, z);
   313    314       sqlite3DbFree(db, z);
   314    315       sqlite3ChangeCookie(pParse, iDb);
   315    316       sqlite3VdbeAddParseSchemaOp(v, iDb,

Changes to src/vacuum.c.

   197    197   
   198    198     /* A VACUUM cannot change the pagesize of an encrypted database. */
   199    199   #ifdef SQLITE_HAS_CODEC
   200    200     if( db->nextPagesize ){
   201    201       extern void sqlite3CodecGetKey(sqlite3*, int, void**, int*);
   202    202       int nKey;
   203    203       char *zKey;
   204         -    sqlite3CodecGetKey(db, 0, (void**)&zKey, &nKey);
          204  +    sqlite3CodecGetKey(db, iDb, (void**)&zKey, &nKey);
   205    205       if( nKey ) db->nextPagesize = 0;
   206    206     }
   207    207   #endif
   208    208   
   209    209     sqlite3BtreeSetCacheSize(pTemp, db->aDb[iDb].pSchema->cache_size);
   210    210     sqlite3BtreeSetSpillSize(pTemp, sqlite3BtreeSetSpillSize(pMain,0));
   211    211     sqlite3BtreeSetPagerFlags(pTemp, PAGER_SYNCHRONOUS_OFF|PAGER_CACHESPILL);

Changes to src/vdbe.c.

   569    569     int rc = SQLITE_OK;        /* Value to return */
   570    570     sqlite3 *db = p->db;       /* The database */
   571    571     u8 resetSchemaOnFault = 0; /* Reset schema after an error if positive */
   572    572     u8 encoding = ENC(db);     /* The database encoding */
   573    573     int iCompare = 0;          /* Result of last comparison */
   574    574     unsigned nVmStep = 0;      /* Number of virtual machine steps */
   575    575   #ifndef SQLITE_OMIT_PROGRESS_CALLBACK
   576         -  unsigned nProgressLimit = 0;/* Invoke xProgress() when nVmStep reaches this */
          576  +  unsigned nProgressLimit;   /* Invoke xProgress() when nVmStep reaches this */
   577    577   #endif
   578    578     Mem *aMem = p->aMem;       /* Copy of p->aMem */
   579    579     Mem *pIn1 = 0;             /* 1st input operand */
   580    580     Mem *pIn2 = 0;             /* 2nd input operand */
   581    581     Mem *pIn3 = 0;             /* 3rd input operand */
   582    582     Mem *pOut = 0;             /* Output operand */
   583    583   #ifdef VDBE_PROFILE
................................................................................
   602    602     if( db->u1.isInterrupted ) goto abort_due_to_interrupt;
   603    603     sqlite3VdbeIOTraceSql(p);
   604    604   #ifndef SQLITE_OMIT_PROGRESS_CALLBACK
   605    605     if( db->xProgress ){
   606    606       u32 iPrior = p->aCounter[SQLITE_STMTSTATUS_VM_STEP];
   607    607       assert( 0 < db->nProgressOps );
   608    608       nProgressLimit = db->nProgressOps - (iPrior % db->nProgressOps);
          609  +  }else{
          610  +    nProgressLimit = 0xffffffff;
   609    611     }
   610    612   #endif
   611    613   #ifdef SQLITE_DEBUG
   612    614     sqlite3BeginBenignMalloc();
   613    615     if( p->pc==0
   614    616      && (p->db->flags & (SQLITE_VdbeListing|SQLITE_VdbeEQP|SQLITE_VdbeTrace))!=0
   615    617     ){
................................................................................
   779    781   #ifndef SQLITE_OMIT_PROGRESS_CALLBACK
   780    782     /* Call the progress callback if it is configured and the required number
   781    783     ** of VDBE ops have been executed (either since this invocation of
   782    784     ** sqlite3VdbeExec() or since last time the progress callback was called).
   783    785     ** If the progress callback returns non-zero, exit the virtual machine with
   784    786     ** a return code SQLITE_ABORT.
   785    787     */
   786         -  if( db->xProgress!=0 && nVmStep>=nProgressLimit ){
          788  +  if( nVmStep>=nProgressLimit && db->xProgress!=0 ){
   787    789       assert( db->nProgressOps!=0 );
   788    790       nProgressLimit = nVmStep + db->nProgressOps - (nVmStep%db->nProgressOps);
   789    791       if( db->xProgress(db->pProgressArg) ){
   790    792         rc = SQLITE_INTERRUPT;
   791    793         goto abort_due_to_error;
   792    794       }
   793    795     }
................................................................................
  1321   1323     assert( pOp->p1>0 );
  1322   1324     assert( pOp->p1+pOp->p2<=(p->nMem+1 - p->nCursor)+1 );
  1323   1325   
  1324   1326   #ifndef SQLITE_OMIT_PROGRESS_CALLBACK
  1325   1327     /* Run the progress counter just before returning.
  1326   1328     */
  1327   1329     if( db->xProgress!=0
  1328         -   && nVmStep>=nProgressLimit
         1330  +   && nVmStep>=nProgressLimit 
  1329   1331      && db->xProgress(db->pProgressArg)!=0
  1330   1332     ){
  1331   1333       rc = SQLITE_INTERRUPT;
  1332   1334       goto abort_due_to_error;
  1333   1335     }
  1334   1336   #endif
  1335   1337   
................................................................................
  2492   2494     u32 avail;         /* Number of bytes of available data */
  2493   2495     u32 t;             /* A type code from the record header */
  2494   2496     Mem *pReg;         /* PseudoTable input register */
  2495   2497   
  2496   2498     pC = p->apCsr[pOp->p1];
  2497   2499     p2 = pOp->p2;
  2498   2500   
  2499         -  /* If the cursor cache is stale, bring it up-to-date */
         2501  +  /* If the cursor cache is stale (meaning it is not currently point at
         2502  +  ** the correct row) then bring it up-to-date by doing the necessary 
         2503  +  ** B-Tree seek. */
  2500   2504     rc = sqlite3VdbeCursorMoveto(&pC, &p2);
  2501   2505     if( rc ) goto abort_due_to_error;
  2502   2506   
  2503   2507     assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  2504   2508     pDest = &aMem[pOp->p3];
  2505   2509     memAboutToChange(p, pDest);
  2506   2510     assert( pOp->p1>=0 && pOp->p1<p->nCursor );
................................................................................
  3974   3978     pC->cacheStatus = CACHE_STALE;
  3975   3979   #ifdef SQLITE_TEST
  3976   3980     sqlite3_search_count++;
  3977   3981   #endif
  3978   3982     if( oc>=OP_SeekGE ){  assert( oc==OP_SeekGE || oc==OP_SeekGT );
  3979   3983       if( res<0 || (res==0 && oc==OP_SeekGT) ){
  3980   3984         res = 0;
  3981         -      rc = sqlite3BtreeNext(pC->uc.pCursor, &res);
  3982         -      if( rc!=SQLITE_OK ) goto abort_due_to_error;
         3985  +      rc = sqlite3BtreeNext(pC->uc.pCursor, 0);
         3986  +      if( rc!=SQLITE_OK ){
         3987  +        if( rc==SQLITE_DONE ){
         3988  +          rc = SQLITE_OK;
         3989  +          res = 1;
         3990  +        }else{
         3991  +          goto abort_due_to_error;
         3992  +        }
         3993  +      }
  3983   3994       }else{
  3984   3995         res = 0;
  3985   3996       }
  3986   3997     }else{
  3987   3998       assert( oc==OP_SeekLT || oc==OP_SeekLE );
  3988   3999       if( res>0 || (res==0 && oc==OP_SeekLT) ){
  3989   4000         res = 0;
  3990         -      rc = sqlite3BtreePrevious(pC->uc.pCursor, &res);
  3991         -      if( rc!=SQLITE_OK ) goto abort_due_to_error;
         4001  +      rc = sqlite3BtreePrevious(pC->uc.pCursor, 0);
         4002  +      if( rc!=SQLITE_OK ){
         4003  +        if( rc==SQLITE_DONE ){
         4004  +          rc = SQLITE_OK;
         4005  +          res = 1;
         4006  +        }else{
         4007  +          goto abort_due_to_error;
         4008  +        }
         4009  +      }
  3992   4010       }else{
  3993   4011         /* res might be negative because the table is empty.  Check to
  3994   4012         ** see if this is the case.
  3995   4013         */
  3996   4014         res = sqlite3BtreeEof(pC->uc.pCursor);
  3997   4015       }
  3998   4016     }
................................................................................
  5090   5108   ** This opcode works just like OP_Next except that P1 must be a
  5091   5109   ** sorter object for which the OP_SorterSort opcode has been
  5092   5110   ** invoked.  This opcode advances the cursor to the next sorted
  5093   5111   ** record, or jumps to P2 if there are no more sorted records.
  5094   5112   */
  5095   5113   case OP_SorterNext: {  /* jump */
  5096   5114     VdbeCursor *pC;
  5097         -  int res;
  5098   5115   
  5099   5116     pC = p->apCsr[pOp->p1];
  5100   5117     assert( isSorter(pC) );
  5101         -  res = 0;
  5102         -  rc = sqlite3VdbeSorterNext(db, pC, &res);
         5118  +  rc = sqlite3VdbeSorterNext(db, pC);
  5103   5119     goto next_tail;
  5104   5120   case OP_PrevIfOpen:    /* jump */
  5105   5121   case OP_NextIfOpen:    /* jump */
  5106   5122     if( p->apCsr[pOp->p1]==0 ) break;
  5107   5123     /* Fall through */
  5108   5124   case OP_Prev:          /* jump */
  5109   5125   case OP_Next:          /* jump */
  5110   5126     assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  5111   5127     assert( pOp->p5<ArraySize(p->aCounter) );
  5112   5128     pC = p->apCsr[pOp->p1];
  5113         -  res = pOp->p3;
  5114   5129     assert( pC!=0 );
  5115   5130     assert( pC->deferredMoveto==0 );
  5116   5131     assert( pC->eCurType==CURTYPE_BTREE );
  5117         -  assert( res==0 || (res==1 && pC->isTable==0) );
  5118         -  testcase( res==1 );
  5119   5132     assert( pOp->opcode!=OP_Next || pOp->p4.xAdvance==sqlite3BtreeNext );
  5120   5133     assert( pOp->opcode!=OP_Prev || pOp->p4.xAdvance==sqlite3BtreePrevious );
  5121   5134     assert( pOp->opcode!=OP_NextIfOpen || pOp->p4.xAdvance==sqlite3BtreeNext );
  5122   5135     assert( pOp->opcode!=OP_PrevIfOpen || pOp->p4.xAdvance==sqlite3BtreePrevious);
  5123   5136   
  5124   5137     /* The Next opcode is only used after SeekGT, SeekGE, and Rewind.
  5125   5138     ** The Prev opcode is only used after SeekLT, SeekLE, and Last. */
................................................................................
  5126   5139     assert( pOp->opcode!=OP_Next || pOp->opcode!=OP_NextIfOpen
  5127   5140          || pC->seekOp==OP_SeekGT || pC->seekOp==OP_SeekGE
  5128   5141          || pC->seekOp==OP_Rewind || pC->seekOp==OP_Found);
  5129   5142     assert( pOp->opcode!=OP_Prev || pOp->opcode!=OP_PrevIfOpen
  5130   5143          || pC->seekOp==OP_SeekLT || pC->seekOp==OP_SeekLE
  5131   5144          || pC->seekOp==OP_Last );
  5132   5145   
  5133         -  rc = pOp->p4.xAdvance(pC->uc.pCursor, &res);
         5146  +  rc = pOp->p4.xAdvance(pC->uc.pCursor, pOp->p3);
  5134   5147   next_tail:
  5135   5148     pC->cacheStatus = CACHE_STALE;
  5136         -  VdbeBranchTaken(res==0,2);
  5137         -  if( rc ) goto abort_due_to_error;
  5138         -  if( res==0 ){
         5149  +  VdbeBranchTaken(rc==SQLITE_OK,2);
         5150  +  if( rc==SQLITE_OK ){
  5139   5151       pC->nullRow = 0;
  5140   5152       p->aCounter[pOp->p5]++;
  5141   5153   #ifdef SQLITE_TEST
  5142   5154       sqlite3_search_count++;
  5143   5155   #endif
  5144   5156       goto jump_to_p2_and_check_for_interrupt;
  5145         -  }else{
  5146         -    pC->nullRow = 1;
  5147   5157     }
         5158  +  if( rc!=SQLITE_DONE ) goto abort_due_to_error;
         5159  +  rc = SQLITE_OK;
         5160  +  pC->nullRow = 1;
  5148   5161     goto check_for_interrupt;
  5149   5162   }
  5150   5163   
  5151   5164   /* Opcode: IdxInsert P1 P2 P3 P4 P5
  5152   5165   ** Synopsis: key=r[P2]
  5153   5166   **
  5154   5167   ** Register P2 holds an SQL index key made using the
................................................................................
  5251   5264     }
  5252   5265     assert( pC->deferredMoveto==0 );
  5253   5266     pC->cacheStatus = CACHE_STALE;
  5254   5267     pC->seekResult = 0;
  5255   5268     break;
  5256   5269   }
  5257   5270   
  5258         -/* Opcode: Seek P1 * P3 P4 *
  5259         -** Synopsis: Move P3 to P1.rowid
         5271  +/* Opcode: DeferredSeek P1 * P3 P4 *
         5272  +** Synopsis: Move P3 to P1.rowid if needed
  5260   5273   **
  5261   5274   ** P1 is an open index cursor and P3 is a cursor on the corresponding
  5262   5275   ** table.  This opcode does a deferred seek of the P3 table cursor
  5263   5276   ** to the row that corresponds to the current row of P1.
  5264   5277   **
  5265   5278   ** This is a deferred seek.  Nothing actually happens until
  5266   5279   ** the cursor is used to read a record.  That way, if no reads
................................................................................
  5279   5292   **
  5280   5293   ** Write into register P2 an integer which is the last entry in the record at
  5281   5294   ** the end of the index key pointed to by cursor P1.  This integer should be
  5282   5295   ** the rowid of the table entry to which this index entry points.
  5283   5296   **
  5284   5297   ** See also: Rowid, MakeRecord.
  5285   5298   */
  5286         -case OP_Seek:
  5287         -case OP_IdxRowid: {              /* out2 */
  5288         -  VdbeCursor *pC;                /* The P1 index cursor */
  5289         -  VdbeCursor *pTabCur;           /* The P2 table cursor (OP_Seek only) */
  5290         -  i64 rowid;                     /* Rowid that P1 current points to */
         5299  +case OP_DeferredSeek:
         5300  +case OP_IdxRowid: {           /* out2 */
         5301  +  VdbeCursor *pC;             /* The P1 index cursor */
         5302  +  VdbeCursor *pTabCur;        /* The P2 table cursor (OP_DeferredSeek only) */
         5303  +  i64 rowid;                  /* Rowid that P1 current points to */
  5291   5304   
  5292   5305     assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  5293   5306     pC = p->apCsr[pOp->p1];
  5294   5307     assert( pC!=0 );
  5295   5308     assert( pC->eCurType==CURTYPE_BTREE );
  5296   5309     assert( pC->uc.pCursor!=0 );
  5297   5310     assert( pC->isTable==0 );
................................................................................
  5309   5322   
  5310   5323     if( !pC->nullRow ){
  5311   5324       rowid = 0;  /* Not needed.  Only used to silence a warning. */
  5312   5325       rc = sqlite3VdbeIdxRowid(db, pC->uc.pCursor, &rowid);
  5313   5326       if( rc!=SQLITE_OK ){
  5314   5327         goto abort_due_to_error;
  5315   5328       }
  5316         -    if( pOp->opcode==OP_Seek ){
         5329  +    if( pOp->opcode==OP_DeferredSeek ){
  5317   5330         assert( pOp->p3>=0 && pOp->p3<p->nCursor );
  5318   5331         pTabCur = p->apCsr[pOp->p3];
  5319   5332         assert( pTabCur!=0 );
  5320   5333         assert( pTabCur->eCurType==CURTYPE_BTREE );
  5321   5334         assert( pTabCur->uc.pCursor!=0 );
  5322   5335         assert( pTabCur->isTable );
  5323   5336         pTabCur->nullRow = 0;

Changes to src/vdbe.h.

    59     59       KeyInfo *pKeyInfo;     /* Used when p4type is P4_KEYINFO */
    60     60       int *ai;               /* Used when p4type is P4_INTARRAY */
    61     61       SubProgram *pProgram;  /* Used when p4type is P4_SUBPROGRAM */
    62     62       Table *pTab;           /* Used when p4type is P4_TABLE */
    63     63   #ifdef SQLITE_ENABLE_CURSOR_HINTS
    64     64       Expr *pExpr;           /* Used when p4type is P4_EXPR */
    65     65   #endif
    66         -    int (*xAdvance)(BtCursor *, int *);
           66  +    int (*xAdvance)(BtCursor *, int);
    67     67     } p4;
    68     68   #ifdef SQLITE_ENABLE_EXPLAIN_COMMENTS
    69     69     char *zComment;          /* Comment to improve readability */
    70     70   #endif
    71     71   #ifdef VDBE_PROFILE
    72     72     u32 cnt;                 /* Number of times this instruction was executed */
    73     73     u64 cycles;              /* Total time spent executing this instruction */

Changes to src/vdbeInt.h.

   502    502   #endif
   503    503   int sqlite3VdbeTransferError(Vdbe *p);
   504    504   
   505    505   int sqlite3VdbeSorterInit(sqlite3 *, int, VdbeCursor *);
   506    506   void sqlite3VdbeSorterReset(sqlite3 *, VdbeSorter *);
   507    507   void sqlite3VdbeSorterClose(sqlite3 *, VdbeCursor *);
   508    508   int sqlite3VdbeSorterRowkey(const VdbeCursor *, Mem *);
   509         -int sqlite3VdbeSorterNext(sqlite3 *, const VdbeCursor *, int *);
          509  +int sqlite3VdbeSorterNext(sqlite3 *, const VdbeCursor *);
   510    510   int sqlite3VdbeSorterRewind(const VdbeCursor *, int *);
   511    511   int sqlite3VdbeSorterWrite(const VdbeCursor *, Mem *);
   512    512   int sqlite3VdbeSorterCompare(const VdbeCursor *, Mem *, int, int *);
   513    513   
   514    514   #if !defined(SQLITE_OMIT_SHARED_CACHE) 
   515    515     void sqlite3VdbeEnter(Vdbe*);
   516    516   #else

Changes to src/vdbeapi.c.

  1315   1315     if( rc==SQLITE_OK ){
  1316   1316       if( zData!=0 ){
  1317   1317         pVar = &p->aVar[i-1];
  1318   1318         rc = sqlite3VdbeMemSetStr(pVar, zData, nData, encoding, xDel);
  1319   1319         if( rc==SQLITE_OK && encoding!=0 ){
  1320   1320           rc = sqlite3VdbeChangeEncoding(pVar, ENC(p->db));
  1321   1321         }
  1322         -      sqlite3Error(p->db, rc);
  1323         -      rc = sqlite3ApiExit(p->db, rc);
         1322  +      if( rc ){
         1323  +        sqlite3Error(p->db, rc);
         1324  +        rc = sqlite3ApiExit(p->db, rc);
         1325  +      }
  1324   1326       }
  1325   1327       sqlite3_mutex_leave(p->db->mutex);
  1326   1328     }else if( xDel!=SQLITE_STATIC && xDel!=SQLITE_TRANSIENT ){
  1327   1329       xDel((void*)zData);
  1328   1330     }
  1329   1331     return rc;
  1330   1332   }
................................................................................
  1650   1652     u32 v;
  1651   1653   #ifdef SQLITE_ENABLE_API_ARMOR
  1652   1654     if( !pStmt ){
  1653   1655       (void)SQLITE_MISUSE_BKPT;
  1654   1656       return 0;
  1655   1657     }
  1656   1658   #endif
  1657         -  v = pVdbe->aCounter[op];
  1658         -  if( resetFlag ) pVdbe->aCounter[op] = 0;
         1659  +  if( op==SQLITE_STMTSTATUS_MEMUSED ){
         1660  +    sqlite3 *db = pVdbe->db;
         1661  +    sqlite3_mutex_enter(db->mutex);
         1662  +    v = 0;
         1663  +    db->pnBytesFreed = (int*)&v;
         1664  +    sqlite3VdbeClearObject(db, pVdbe);
         1665  +    sqlite3DbFree(db, pVdbe);
         1666  +    db->pnBytesFreed = 0;
         1667  +    sqlite3_mutex_leave(db->mutex);
         1668  +  }else{
         1669  +    v = pVdbe->aCounter[op];
         1670  +    if( resetFlag ) pVdbe->aCounter[op] = 0;
         1671  +  }
  1659   1672     return (int)v;
  1660   1673   }
  1661   1674   
  1662   1675   /*
  1663   1676   ** Return the SQL associated with a prepared statement
  1664   1677   */
  1665   1678   const char *sqlite3_sql(sqlite3_stmt *pStmt){

Changes to src/vdbeaux.c.

  2156   2156   /*
  2157   2157   ** Set the number of result columns that will be returned by this SQL
  2158   2158   ** statement. This is now set at compile time, rather than during
  2159   2159   ** execution of the vdbe program so that sqlite3_column_count() can
  2160   2160   ** be called on an SQL statement before sqlite3_step().
  2161   2161   */
  2162   2162   void sqlite3VdbeSetNumCols(Vdbe *p, int nResColumn){
  2163         -  Mem *pColName;
  2164   2163     int n;
  2165   2164     sqlite3 *db = p->db;
  2166   2165   
  2167         -  releaseMemArray(p->aColName, p->nResColumn*COLNAME_N);
  2168         -  sqlite3DbFree(db, p->aColName);
         2166  +  if( p->nResColumn ){
         2167  +    releaseMemArray(p->aColName, p->nResColumn*COLNAME_N);
         2168  +    sqlite3DbFree(db, p->aColName);
         2169  +  }
  2169   2170     n = nResColumn*COLNAME_N;
  2170   2171     p->nResColumn = (u16)nResColumn;
  2171         -  p->aColName = pColName = (Mem*)sqlite3DbMallocRawNN(db, sizeof(Mem)*n );
         2172  +  p->aColName = (Mem*)sqlite3DbMallocRawNN(db, sizeof(Mem)*n );
  2172   2173     if( p->aColName==0 ) return;
  2173         -  initMemArray(p->aColName, n, p->db, MEM_Null);
         2174  +  initMemArray(p->aColName, n, db, MEM_Null);
  2174   2175   }
  2175   2176   
  2176   2177   /*
  2177   2178   ** Set the name of the idx'th column to be returned by the SQL statement.
  2178   2179   ** zName must be a pointer to a nul terminated string.
  2179   2180   **
  2180   2181   ** This call must be made after a call to sqlite3VdbeSetNumCols().
................................................................................
  2816   2817     if( p->zErrMsg ){
  2817   2818       db->bBenignMalloc++;
  2818   2819       sqlite3BeginBenignMalloc();
  2819   2820       if( db->pErr==0 ) db->pErr = sqlite3ValueNew(db);
  2820   2821       sqlite3ValueSetStr(db->pErr, -1, p->zErrMsg, SQLITE_UTF8, SQLITE_TRANSIENT);
  2821   2822       sqlite3EndBenignMalloc();
  2822   2823       db->bBenignMalloc--;
  2823         -    db->errCode = rc;
  2824         -  }else{
  2825         -    sqlite3Error(db, rc);
         2824  +  }else if( db->pErr ){
         2825  +    sqlite3ValueSetNull(db->pErr);
  2826   2826     }
         2827  +  db->errCode = rc;
  2827   2828     return rc;
  2828   2829   }
  2829   2830   
  2830   2831   #ifdef SQLITE_ENABLE_SQLLOG
  2831   2832   /*
  2832   2833   ** If an SQLITE_CONFIG_SQLLOG hook is registered and the VM has been run, 
  2833   2834   ** invoke it.
................................................................................
  3727   3728     if( pMem1->enc==pColl->enc ){
  3728   3729       /* The strings are already in the correct encoding.  Call the
  3729   3730        ** comparison function directly */
  3730   3731       return pColl->xCmp(pColl->pUser,pMem1->n,pMem1->z,pMem2->n,pMem2->z);
  3731   3732     }else{
  3732   3733       int rc;
  3733   3734       const void *v1, *v2;
  3734         -    int n1, n2;
  3735   3735       Mem c1;
  3736   3736       Mem c2;
  3737   3737       sqlite3VdbeMemInit(&c1, pMem1->db, MEM_Null);
  3738   3738       sqlite3VdbeMemInit(&c2, pMem1->db, MEM_Null);
  3739   3739       sqlite3VdbeMemShallowCopy(&c1, pMem1, MEM_Ephem);
  3740   3740       sqlite3VdbeMemShallowCopy(&c2, pMem2, MEM_Ephem);
  3741   3741       v1 = sqlite3ValueText((sqlite3_value*)&c1, pColl->enc);
  3742         -    n1 = v1==0 ? 0 : c1.n;
  3743   3742       v2 = sqlite3ValueText((sqlite3_value*)&c2, pColl->enc);
  3744         -    n2 = v2==0 ? 0 : c2.n;
  3745         -    rc = pColl->xCmp(pColl->pUser, n1, v1, n2, v2);
  3746         -    if( (v1==0 || v2==0) && prcErr ) *prcErr = SQLITE_NOMEM_BKPT;
         3743  +    if( (v1==0 || v2==0) ){
         3744  +      if( prcErr ) *prcErr = SQLITE_NOMEM_BKPT;
         3745  +      rc = 0;
         3746  +    }else{
         3747  +      rc = pColl->xCmp(pColl->pUser, c1.n, v1, c2.n, v2);
         3748  +    }
  3747   3749       sqlite3VdbeMemRelease(&c1);
  3748   3750       sqlite3VdbeMemRelease(&c2);
  3749   3751       return rc;
  3750   3752     }
  3751   3753   }
  3752   3754   
  3753   3755   /*

Changes to src/vdbemem.c.

  1321   1321       }
  1322   1322       if( pVal->flags & (MEM_Int|MEM_Real) ) pVal->flags &= ~MEM_Str;
  1323   1323       if( enc!=SQLITE_UTF8 ){
  1324   1324         rc = sqlite3VdbeChangeEncoding(pVal, enc);
  1325   1325       }
  1326   1326     }else if( op==TK_UMINUS ) {
  1327   1327       /* This branch happens for multiple negative signs.  Ex: -(-5) */
  1328         -    if( SQLITE_OK==sqlite3ValueFromExpr(db,pExpr->pLeft,enc,affinity,&pVal) 
         1328  +    if( SQLITE_OK==valueFromExpr(db,pExpr->pLeft,enc,affinity,&pVal,pCtx) 
  1329   1329        && pVal!=0
  1330   1330       ){
  1331   1331         sqlite3VdbeMemNumerify(pVal);
  1332   1332         if( pVal->flags & MEM_Real ){
  1333   1333           pVal->u.r = -pVal->u.r;
  1334   1334         }else if( pVal->u.i==SMALLEST_INT64 ){
  1335   1335           pVal->u.r = -(double)SMALLEST_INT64;

Changes to src/vdbesort.c.

   811    811     const u8 * const v1 = &p1[ p1[0] ];   /* Pointer to value 1 */
   812    812     const u8 * const v2 = &p2[ p2[0] ];   /* Pointer to value 2 */
   813    813   
   814    814     int n1;
   815    815     int n2;
   816    816     int res;
   817    817   
   818         -  getVarint32(&p1[1], n1); n1 = (n1 - 13) / 2;
   819         -  getVarint32(&p2[1], n2); n2 = (n2 - 13) / 2;
   820         -  res = memcmp(v1, v2, MIN(n1, n2));
          818  +  getVarint32(&p1[1], n1);
          819  +  getVarint32(&p2[1], n2);
          820  +  res = memcmp(v1, v2, (MIN(n1, n2) - 13)/2);
   821    821     if( res==0 ){
   822    822       res = n1 - n2;
   823    823     }
   824    824   
   825    825     if( res==0 ){
   826    826       if( pTask->pSorter->pKeyInfo->nField>1 ){
   827    827         res = vdbeSorterCompareTail(
................................................................................
  2608   2608     }
  2609   2609   
  2610   2610     vdbeSorterRewindDebug("rewinddone");
  2611   2611     return rc;
  2612   2612   }
  2613   2613   
  2614   2614   /*
  2615         -** Advance to the next element in the sorter.
         2615  +** Advance to the next element in the sorter.  Return value:
         2616  +**
         2617  +**    SQLITE_OK     success
         2618  +**    SQLITE_DONE   end of data
         2619  +**    otherwise     some kind of error.
  2616   2620   */
  2617         -int sqlite3VdbeSorterNext(sqlite3 *db, const VdbeCursor *pCsr, int *pbEof){
         2621  +int sqlite3VdbeSorterNext(sqlite3 *db, const VdbeCursor *pCsr){
  2618   2622     VdbeSorter *pSorter;
  2619   2623     int rc;                         /* Return code */
  2620   2624   
  2621   2625     assert( pCsr->eCurType==CURTYPE_SORTER );
  2622   2626     pSorter = pCsr->uc.pSorter;
  2623   2627     assert( pSorter->bUsePMA || (pSorter->pReader==0 && pSorter->pMerger==0) );
  2624   2628     if( pSorter->bUsePMA ){
  2625   2629       assert( pSorter->pReader==0 || pSorter->pMerger==0 );
  2626   2630       assert( pSorter->bUseThreads==0 || pSorter->pReader );
  2627   2631       assert( pSorter->bUseThreads==1 || pSorter->pMerger );
  2628   2632   #if SQLITE_MAX_WORKER_THREADS>0
  2629   2633       if( pSorter->bUseThreads ){
  2630   2634         rc = vdbePmaReaderNext(pSorter->pReader);
  2631         -      *pbEof = (pSorter->pReader->pFd==0);
         2635  +      if( rc==SQLITE_OK && pSorter->pReader->pFd==0 ) rc = SQLITE_DONE;
  2632   2636       }else
  2633   2637   #endif
  2634   2638       /*if( !pSorter->bUseThreads )*/ {
         2639  +      int res = 0;
  2635   2640         assert( pSorter->pMerger!=0 );
  2636   2641         assert( pSorter->pMerger->pTask==(&pSorter->aTask[0]) );
  2637         -      rc = vdbeMergeEngineStep(pSorter->pMerger, pbEof);
         2642  +      rc = vdbeMergeEngineStep(pSorter->pMerger, &res);
         2643  +      if( rc==SQLITE_OK && res ) rc = SQLITE_DONE;
  2638   2644       }
  2639   2645     }else{
  2640   2646       SorterRecord *pFree = pSorter->list.pList;
  2641   2647       pSorter->list.pList = pFree->u.pNext;
  2642   2648       pFree->u.pNext = 0;
  2643   2649       if( pSorter->list.aMemory==0 ) vdbeSorterRecordFree(db, pFree);
  2644         -    *pbEof = !pSorter->list.pList;
  2645         -    rc = SQLITE_OK;
         2650  +    rc = pSorter->list.pList ? SQLITE_OK : SQLITE_DONE;
  2646   2651     }
  2647   2652     return rc;
  2648   2653   }
  2649   2654   
  2650   2655   /*
  2651   2656   ** Return a pointer to a buffer owned by the sorter that contains the 
  2652   2657   ** current key.

Changes to src/walker.c.

   120    120   /*
   121    121   ** Call sqlite3WalkExpr() for every expression in Select statement p.
   122    122   ** Invoke sqlite3WalkSelect() for subqueries in the FROM clause and
   123    123   ** on the compound select chain, p->pPrior. 
   124    124   **
   125    125   ** If it is not NULL, the xSelectCallback() callback is invoked before
   126    126   ** the walk of the expressions and FROM clause. The xSelectCallback2()
   127         -** method, if it is not NULL, is invoked following the walk of the 
   128         -** expressions and FROM clause.
          127  +** method is invoked following the walk of the expressions and FROM clause,
          128  +** but only if both xSelectCallback and xSelectCallback2 are both non-NULL
          129  +** and if the expressions and FROM clause both return WRC_Continue;
   129    130   **
   130    131   ** Return WRC_Continue under normal conditions.  Return WRC_Abort if
   131    132   ** there is an abort request.
   132    133   **
   133    134   ** If the Walker does not have an xSelectCallback() then this routine
   134    135   ** is a no-op returning WRC_Continue.
   135    136   */
   136    137   int sqlite3WalkSelect(Walker *pWalker, Select *p){
   137    138     int rc;
   138         -  if( p==0 || (pWalker->xSelectCallback==0 && pWalker->xSelectCallback2==0) ){
   139         -    return WRC_Continue;
   140         -  }
   141         -  rc = WRC_Continue;
   142         -  pWalker->walkerDepth++;
   143         -  while( p ){
   144         -    if( pWalker->xSelectCallback ){
   145         -       rc = pWalker->xSelectCallback(pWalker, p);
   146         -       if( rc ) break;
   147         -    }
          139  +  if( p==0 || pWalker->xSelectCallback==0 ) return WRC_Continue;
          140  +  do{
          141  +    rc = pWalker->xSelectCallback(pWalker, p);
          142  +    if( rc ) return rc & WRC_Abort;
   148    143       if( sqlite3WalkSelectExpr(pWalker, p)
   149    144        || sqlite3WalkSelectFrom(pWalker, p)
   150    145       ){
   151         -      pWalker->walkerDepth--;
   152    146         return WRC_Abort;
   153    147       }
   154    148       if( pWalker->xSelectCallback2 ){
   155    149         pWalker->xSelectCallback2(pWalker, p);
   156    150       }
   157    151       p = p->pPrior;
   158         -  }
   159         -  pWalker->walkerDepth--;
   160         -  return rc & WRC_Abort;
          152  +  }while( p!=0 );
          153  +  return WRC_Continue;
   161    154   }

Changes to src/where.c.

  1188   1188       }
  1189   1189       if( roundUp ){
  1190   1190         iGap = (iGap*2)/3;
  1191   1191       }else{
  1192   1192         iGap = iGap/3;
  1193   1193       }
  1194   1194       aStat[0] = iLower + iGap;
  1195         -    aStat[1] = pIdx->aAvgEq[iCol];
         1195  +    aStat[1] = pIdx->aAvgEq[nField-1];
  1196   1196     }
  1197   1197   
  1198   1198     /* Restore the pRec->nField value before returning.  */
  1199   1199     pRec->nField = nField;
  1200   1200     return i;
  1201   1201   }
  1202   1202   #endif /* SQLITE_ENABLE_STAT3_OR_STAT4 */
................................................................................
  1941   1941         pTemplate->nOut = p->nOut + 1;
  1942   1942       }
  1943   1943     }
  1944   1944   }
  1945   1945   
  1946   1946   /*
  1947   1947   ** Search the list of WhereLoops in *ppPrev looking for one that can be
  1948         -** supplanted by pTemplate.
         1948  +** replaced by pTemplate.
  1949   1949   **
  1950         -** Return NULL if the WhereLoop list contains an entry that can supplant
  1951         -** pTemplate, in other words if pTemplate does not belong on the list.
         1950  +** Return NULL if pTemplate does not belong on the WhereLoop list.
         1951  +** In other words if pTemplate ought to be dropped from further consideration.
  1952   1952   **
  1953         -** If pX is a WhereLoop that pTemplate can supplant, then return the
         1953  +** If pX is a WhereLoop that pTemplate can replace, then return the
  1954   1954   ** link that points to pX.
  1955   1955   **
  1956         -** If pTemplate cannot supplant any existing element of the list but needs
  1957         -** to be added to the list, then return a pointer to the tail of the list.
         1956  +** If pTemplate cannot replace any existing element of the list but needs
         1957  +** to be added to the list as a new entry, then return a pointer to the
         1958  +** tail of the list.
  1958   1959   */
  1959   1960   static WhereLoop **whereLoopFindLesser(
  1960   1961     WhereLoop **ppPrev,
  1961   1962     const WhereLoop *pTemplate
  1962   1963   ){
  1963   1964     WhereLoop *p;
  1964   1965     for(p=(*ppPrev); p; ppPrev=&p->pNextLoop, p=*ppPrev){
................................................................................
  2095   2096     ** WhereLoop and insert it.
  2096   2097     */
  2097   2098   #if WHERETRACE_ENABLED /* 0x8 */
  2098   2099     if( sqlite3WhereTrace & 0x8 ){
  2099   2100       if( p!=0 ){
  2100   2101         sqlite3DebugPrintf("replace: ");
  2101   2102         whereLoopPrint(p, pBuilder->pWC);
         2103  +      sqlite3DebugPrintf("   with: ");
         2104  +    }else{
         2105  +      sqlite3DebugPrintf("    add: ");
  2102   2106       }
  2103         -    sqlite3DebugPrintf("    add: ");
  2104   2107       whereLoopPrint(pTemplate, pBuilder->pWC);
  2105   2108     }
  2106   2109   #endif
  2107   2110     if( p==0 ){
  2108   2111       /* Allocate a new WhereLoop to add to the end of the list */
  2109   2112       *ppPrev = p = sqlite3DbMallocRawNN(db, sizeof(WhereLoop));
  2110   2113       if( p==0 ) return SQLITE_NOMEM_BKPT;
................................................................................
  3996   3999              && (rCost>mxCost || (rCost==mxCost && rUnsorted>=mxUnsorted))
  3997   4000             ){
  3998   4001               /* The current candidate is no better than any of the mxChoice
  3999   4002               ** paths currently in the best-so-far buffer.  So discard
  4000   4003               ** this candidate as not viable. */
  4001   4004   #ifdef WHERETRACE_ENABLED /* 0x4 */
  4002   4005               if( sqlite3WhereTrace&0x4 ){
  4003         -              sqlite3DebugPrintf("Skip   %s cost=%-3d,%3d order=%c\n",
  4004         -                  wherePathName(pFrom, iLoop, pWLoop), rCost, nOut,
         4006  +              sqlite3DebugPrintf("Skip   %s cost=%-3d,%3d,%3d order=%c\n",
         4007  +                  wherePathName(pFrom, iLoop, pWLoop), rCost, nOut, rUnsorted,
  4005   4008                     isOrdered>=0 ? isOrdered+'0' : '?');
  4006   4009               }
  4007   4010   #endif
  4008   4011               continue;
  4009   4012             }
  4010   4013             /* If we reach this points it means that the new candidate path
  4011   4014             ** needs to be added to the set of best-so-far paths. */
................................................................................
  4015   4018             }else{
  4016   4019               /* New path replaces the prior worst to keep count below mxChoice */
  4017   4020               jj = mxI;
  4018   4021             }
  4019   4022             pTo = &aTo[jj];
  4020   4023   #ifdef WHERETRACE_ENABLED /* 0x4 */
  4021   4024             if( sqlite3WhereTrace&0x4 ){
  4022         -            sqlite3DebugPrintf("New    %s cost=%-3d,%3d order=%c\n",
  4023         -                wherePathName(pFrom, iLoop, pWLoop), rCost, nOut,
         4025  +            sqlite3DebugPrintf("New    %s cost=%-3d,%3d,%3d order=%c\n",
         4026  +                wherePathName(pFrom, iLoop, pWLoop), rCost, nOut, rUnsorted,
  4024   4027                   isOrdered>=0 ? isOrdered+'0' : '?');
  4025   4028             }
  4026   4029   #endif
  4027   4030           }else{
  4028   4031             /* Control reaches here if best-so-far path pTo=aTo[jj] covers the
  4029         -          ** same set of loops and has the sam isOrdered setting as the
         4032  +          ** same set of loops and has the same isOrdered setting as the
  4030   4033             ** candidate path.  Check to see if the candidate should replace
  4031         -          ** pTo or if the candidate should be skipped */
  4032         -          if( pTo->rCost<rCost || (pTo->rCost==rCost && pTo->nRow<=nOut) ){
         4034  +          ** pTo or if the candidate should be skipped.
         4035  +          ** 
         4036  +          ** The conditional is an expanded vector comparison equivalent to:
         4037  +          **   (pTo->rCost,pTo->nRow,pTo->rUnsorted) <= (rCost,nOut,rUnsorted)
         4038  +          */
         4039  +          if( pTo->rCost<rCost 
         4040  +           || (pTo->rCost==rCost
         4041  +               && (pTo->nRow<nOut
         4042  +                   || (pTo->nRow==nOut && pTo->rUnsorted<=rUnsorted)
         4043  +                  )
         4044  +              )
         4045  +          ){
  4033   4046   #ifdef WHERETRACE_ENABLED /* 0x4 */
  4034   4047               if( sqlite3WhereTrace&0x4 ){
  4035   4048                 sqlite3DebugPrintf(
  4036         -                  "Skip   %s cost=%-3d,%3d order=%c",
  4037         -                  wherePathName(pFrom, iLoop, pWLoop), rCost, nOut,
         4049  +                  "Skip   %s cost=%-3d,%3d,%3d order=%c",
         4050  +                  wherePathName(pFrom, iLoop, pWLoop), rCost, nOut, rUnsorted,
  4038   4051                     isOrdered>=0 ? isOrdered+'0' : '?');
  4039         -              sqlite3DebugPrintf("   vs %s cost=%-3d,%d order=%c\n",
         4052  +              sqlite3DebugPrintf("   vs %s cost=%-3d,%3d,%3d order=%c\n",
  4040   4053                     wherePathName(pTo, iLoop+1, 0), pTo->rCost, pTo->nRow,
  4041         -                  pTo->isOrdered>=0 ? pTo->isOrdered+'0' : '?');
         4054  +                  pTo->rUnsorted, pTo->isOrdered>=0 ? pTo->isOrdered+'0' : '?');
  4042   4055               }
  4043   4056   #endif
  4044   4057               /* Discard the candidate path from further consideration */
  4045   4058               testcase( pTo->rCost==rCost );
  4046   4059               continue;
  4047   4060             }
  4048   4061             testcase( pTo->rCost==rCost+1 );
  4049   4062             /* Control reaches here if the candidate path is better than the
  4050   4063             ** pTo path.  Replace pTo with the candidate. */
  4051   4064   #ifdef WHERETRACE_ENABLED /* 0x4 */
  4052   4065             if( sqlite3WhereTrace&0x4 ){
  4053   4066               sqlite3DebugPrintf(
  4054         -                "Update %s cost=%-3d,%3d order=%c",
  4055         -                wherePathName(pFrom, iLoop, pWLoop), rCost, nOut,
         4067  +                "Update %s cost=%-3d,%3d,%3d order=%c",
         4068  +                wherePathName(pFrom, iLoop, pWLoop), rCost, nOut, rUnsorted,
  4056   4069                   isOrdered>=0 ? isOrdered+'0' : '?');
  4057         -            sqlite3DebugPrintf("  was %s cost=%-3d,%3d order=%c\n",
         4070  +            sqlite3DebugPrintf("  was %s cost=%-3d,%3d,%3d order=%c\n",
  4058   4071                   wherePathName(pTo, iLoop+1, 0), pTo->rCost, pTo->nRow,
  4059         -                pTo->isOrdered>=0 ? pTo->isOrdered+'0' : '?');
         4072  +                pTo->rUnsorted, pTo->isOrdered>=0 ? pTo->isOrdered+'0' : '?');
  4060   4073             }
  4061   4074   #endif
  4062   4075           }
  4063   4076           /* pWLoop is a winner.  Add it to the set of best so far */
  4064   4077           pTo->maskLoop = pFrom->maskLoop | pWLoop->maskSelf;
  4065   4078           pTo->revLoop = revMask;
  4066   4079           pTo->nRow = nOut;
................................................................................
  4276   4289   #ifdef SQLITE_DEBUG
  4277   4290       pLoop->cId = '0';
  4278   4291   #endif
  4279   4292       return 1;
  4280   4293     }
  4281   4294     return 0;
  4282   4295   }
         4296  +
         4297  +/*
         4298  +** Helper function for exprIsDeterministic().
         4299  +*/
         4300  +static int exprNodeIsDeterministic(Walker *pWalker, Expr *pExpr){
         4301  +  if( pExpr->op==TK_FUNCTION && ExprHasProperty(pExpr, EP_ConstFunc)==0 ){
         4302  +    pWalker->eCode = 0;
         4303  +    return WRC_Abort;
         4304  +  }
         4305  +  return WRC_Continue;
         4306  +}
         4307  +
         4308  +/*
         4309  +** Return true if the expression contains no non-deterministic SQL 
         4310  +** functions. Do not consider non-deterministic SQL functions that are 
         4311  +** part of sub-select statements.
         4312  +*/
         4313  +static int exprIsDeterministic(Expr *p){
         4314  +  Walker w;
         4315  +  memset(&w, 0, sizeof(w));
         4316  +  w.eCode = 1;
         4317  +  w.xExprCallback = exprNodeIsDeterministic;
         4318  +  sqlite3WalkExpr(&w, p);
         4319  +  return w.eCode;
         4320  +}
  4283   4321   
  4284   4322   /*
  4285   4323   ** Generate the beginning of the loop used for WHERE clause processing.
  4286   4324   ** The return value is a pointer to an opaque structure that contains
  4287   4325   ** information needed to terminate the loop.  Later, the calling routine
  4288   4326   ** should invoke sqlite3WhereEnd() with the return value of this function
  4289   4327   ** in order to complete the WHERE clause processing.
................................................................................
  4475   4513     /* Split the WHERE clause into separate subexpressions where each
  4476   4514     ** subexpression is separated by an AND operator.
  4477   4515     */
  4478   4516     initMaskSet(pMaskSet);
  4479   4517     sqlite3WhereClauseInit(&pWInfo->sWC, pWInfo);
  4480   4518     sqlite3WhereSplit(&pWInfo->sWC, pWhere, TK_AND);
  4481   4519       
  4482         -  /* Special case: a WHERE clause that is constant.  Evaluate the
  4483         -  ** expression and either jump over all of the code or fall thru.
  4484         -  */
  4485         -  for(ii=0; ii<sWLB.pWC->nTerm; ii++){
  4486         -    if( nTabList==0 || sqlite3ExprIsConstantNotJoin(sWLB.pWC->a[ii].pExpr) ){
  4487         -      sqlite3ExprIfFalse(pParse, sWLB.pWC->a[ii].pExpr, pWInfo->iBreak,
  4488         -                         SQLITE_JUMPIFNULL);
  4489         -      sWLB.pWC->a[ii].wtFlags |= TERM_CODED;
  4490         -    }
  4491         -  }
  4492         -
  4493   4520     /* Special case: No FROM clause
  4494   4521     */
  4495   4522     if( nTabList==0 ){
  4496   4523       if( pOrderBy ) pWInfo->nOBSat = pOrderBy->nExpr;
  4497   4524       if( wctrlFlags & WHERE_WANT_DISTINCT ){
  4498   4525         pWInfo->eDistinct = WHERE_DISTINCT_UNIQUE;
  4499   4526       }
................................................................................
  4523   4550       assert( m==MASKBIT(ii) );
  4524   4551     }
  4525   4552   #endif
  4526   4553   
  4527   4554     /* Analyze all of the subexpressions. */
  4528   4555     sqlite3WhereExprAnalyze(pTabList, &pWInfo->sWC);
  4529   4556     if( db->mallocFailed ) goto whereBeginError;
         4557  +
         4558  +  /* Special case: WHERE terms that do not refer to any tables in the join
         4559  +  ** (constant expressions). Evaluate each such term, and jump over all the
         4560  +  ** generated code if the result is not true.  
         4561  +  **
         4562  +  ** Do not do this if the expression contains non-deterministic functions
         4563  +  ** that are not within a sub-select. This is not strictly required, but
         4564  +  ** preserves SQLite's legacy behaviour in the following two cases:
         4565  +  **
         4566  +  **   FROM ... WHERE random()>0;           -- eval random() once per row
         4567  +  **   FROM ... WHERE (SELECT random())>0;  -- eval random() once overall
         4568  +  */
         4569  +  for(ii=0; ii<sWLB.pWC->nTerm; ii++){
         4570  +    WhereTerm *pT = &sWLB.pWC->a[ii];
         4571  +    if( pT->prereqAll==0 && (nTabList==0 || exprIsDeterministic(pT->pExpr)) ){
         4572  +      sqlite3ExprIfFalse(pParse, pT->pExpr, pWInfo->iBreak, SQLITE_JUMPIFNULL);
         4573  +      pT->wtFlags |= TERM_CODED;
         4574  +    }
         4575  +  }
  4530   4576   
  4531   4577     if( wctrlFlags & WHERE_WANT_DISTINCT ){
  4532   4578       if( isDistinctRedundant(pParse, pTabList, &pWInfo->sWC, pResultSet) ){
  4533   4579         /* The DISTINCT marking is pointless.  Ignore it. */
  4534   4580         pWInfo->eDistinct = WHERE_DISTINCT_UNIQUE;
  4535   4581       }else if( pOrderBy==0 ){
  4536   4582         /* Try to ORDER BY the result set to make distinct processing easier */
................................................................................
  4560   4606   #ifdef WHERETRACE_ENABLED
  4561   4607       if( sqlite3WhereTrace ){    /* Display all of the WhereLoop objects */
  4562   4608         WhereLoop *p;
  4563   4609         int i;
  4564   4610         static const char zLabel[] = "0123456789abcdefghijklmnopqrstuvwyxz"
  4565   4611                                                "ABCDEFGHIJKLMNOPQRSTUVWYXZ";
  4566   4612         for(p=pWInfo->pLoops, i=0; p; p=p->pNextLoop, i++){
  4567         -        p->cId = zLabel[i%sizeof(zLabel)];
         4613  +        p->cId = zLabel[i%(sizeof(zLabel)-1)];
  4568   4614           whereLoopPrint(p, sWLB.pWC);
  4569   4615         }
  4570   4616       }
  4571   4617   #endif
  4572   4618     
  4573   4619       wherePathSolver(pWInfo, 0);
  4574   4620       if( db->mallocFailed ) goto whereBeginError;

Changes to src/wherecode.c.

   962    962   ** Cursor iCur is open on an intkey b-tree (a table). Register iRowid contains
   963    963   ** a rowid value just read from cursor iIdxCur, open on index pIdx. This
   964    964   ** function generates code to do a deferred seek of cursor iCur to the 
   965    965   ** rowid stored in register iRowid.
   966    966   **
   967    967   ** Normally, this is just:
   968    968   **
   969         -**   OP_Seek $iCur $iRowid
          969  +**   OP_DeferredSeek $iCur $iRowid
   970    970   **
   971    971   ** However, if the scan currently being coded is a branch of an OR-loop and
   972         -** the statement currently being coded is a SELECT, then P3 of the OP_Seek
          972  +** the statement currently being coded is a SELECT, then P3 of OP_DeferredSeek
   973    973   ** is set to iIdxCur and P4 is set to point to an array of integers
   974    974   ** containing one entry for each column of the table cursor iCur is open 
   975    975   ** on. For each table column, if the column is the i'th column of the 
   976    976   ** index, then the corresponding array entry is set to (i+1). If the column
   977    977   ** does not appear in the index at all, the array entry is set to 0.
   978    978   */
   979    979   static void codeDeferredSeek(
................................................................................
   984    984   ){
   985    985     Parse *pParse = pWInfo->pParse; /* Parse context */
   986    986     Vdbe *v = pParse->pVdbe;        /* Vdbe to generate code within */
   987    987   
   988    988     assert( iIdxCur>0 );
   989    989     assert( pIdx->aiColumn[pIdx->nColumn-1]==-1 );
   990    990     
   991         -  sqlite3VdbeAddOp3(v, OP_Seek, iIdxCur, 0, iCur);
          991  +  sqlite3VdbeAddOp3(v, OP_DeferredSeek, iIdxCur, 0, iCur);
   992    992     if( (pWInfo->wctrlFlags & WHERE_OR_SUBCLAUSE)
   993    993      && DbMaskAllZero(sqlite3ParseToplevel(pParse)->writeMask)
   994    994     ){
   995    995       int i;
   996    996       Table *pTab = pIdx->pTable;
   997    997       int *ai = (int*)sqlite3DbMallocZero(pParse->db, sizeof(int)*(pTab->nCol+1));
   998    998       if( ai ){

Changes to test/ctime.test.

    56     56   # the results should be in sorted order already
    57     57   do_test ctime-1.2.2 {
    58     58     set ans [ catchsql {
    59     59       PRAGMA compile_options;
    60     60     } ]
    61     61     list [ lindex $ans 0 ] [ expr { [lsort [lindex $ans 1]]==[lindex $ans 1] } ]
    62     62   } {0 1}
           63  +
           64  +# Check the THREADSAFE option for SQLITE_THREADSAFE=2 builds (there are
           65  +# a couple of these configurations in releasetest.tcl).
           66  +#
           67  +ifcapable threadsafe2 {
           68  +  foreach {tn opt res} {
           69  +    1 SQLITE_THREADSAFE     1
           70  +    2 THREADSAFE            1
           71  +    3 THREADSAFE=0          0
           72  +    4 THREADSAFE=1          0
           73  +    5 THREADSAFE=2          1
           74  +    6 THREADSAFE=           0
           75  +  } {
           76  +    do_execsql_test ctime-1.3.$tn {
           77  +      SELECT sqlite_compileoption_used($opt)
           78  +    } $res
           79  +  }
           80  +}
    63     81   
    64     82   # SQLITE_THREADSAFE should pretty much always be defined
    65     83   # one way or the other, and it must have a value of 0 or 1.
    66     84   do_test ctime-1.4.1 {
    67     85     catchsql {
    68     86       SELECT sqlite_compileoption_used('SQLITE_THREADSAFE');
    69     87     }

Changes to test/eqp.test.

   184    184     0 0 0 {SCAN TABLE t1}
   185    185     0 0 0 {EXECUTE SCALAR SUBQUERY 1}
   186    186     1 0 0 {SCAN TABLE t1 AS sub}
   187    187   }
   188    188   do_eqp_test 3.1.2 {
   189    189     SELECT * FROM t1 WHERE (SELECT x FROM t1 AS sub);
   190    190   } {
   191         -  0 0 0 {SCAN TABLE t1}
   192    191     0 0 0 {EXECUTE SCALAR SUBQUERY 1}
   193    192     1 0 0 {SCAN TABLE t1 AS sub}
          193  +  0 0 0 {SCAN TABLE t1}
   194    194   }
   195    195   do_eqp_test 3.1.3 {
   196    196     SELECT * FROM t1 WHERE (SELECT x FROM t1 AS sub ORDER BY y);
   197    197   } {
   198         -  0 0 0 {SCAN TABLE t1}
   199    198     0 0 0 {EXECUTE SCALAR SUBQUERY 1}
   200    199     1 0 0 {SCAN TABLE t1 AS sub}
   201    200     1 0 0 {USE TEMP B-TREE FOR ORDER BY}
          201  +  0 0 0 {SCAN TABLE t1}
   202    202   }
   203    203   do_eqp_test 3.1.4 {
   204    204     SELECT * FROM t1 WHERE (SELECT x FROM t2 ORDER BY x);
   205    205   } {
   206         -  0 0 0 {SCAN TABLE t1}
   207    206     0 0 0 {EXECUTE SCALAR SUBQUERY 1}
   208    207     1 0 0 {SCAN TABLE t2 USING COVERING INDEX t2i1}
          208  +  0 0 0 {SCAN TABLE t1}
   209    209   }
   210    210   
   211    211   det 3.2.1 {
   212    212     SELECT * FROM (SELECT * FROM t1 ORDER BY x LIMIT 10) ORDER BY y LIMIT 5
   213    213   } {
   214    214     1 0 0 {SCAN TABLE t1} 
   215    215     1 0 0 {USE TEMP B-TREE FOR ORDER BY} 

Changes to test/fts4unicode.test.

   380    380     do_isspace_test 6.$T.9 $T    8196
   381    381     do_isspace_test 6.$T.10 $T    8197
   382    382     do_isspace_test 6.$T.11 $T    8198
   383    383     do_isspace_test 6.$T.12 $T    8199
   384    384     do_isspace_test 6.$T.13 $T    8200
   385    385     do_isspace_test 6.$T.14 $T    8201
   386    386     do_isspace_test 6.$T.15 $T    8202
   387         -  do_isspace_test 6.$T.16 $T    8239
          387  +  if {$T!="icu"} {
          388  +    do_isspace_test 6.$T.16 $T    8239
          389  +  }
   388    390     do_isspace_test 6.$T.17 $T    8287
   389    391     do_isspace_test 6.$T.18 $T   12288
   390    392   
   391    393     if {$T!="icu"} {
   392    394       do_isspace_test 6.$T.19 $T   {32 160 5760 6158}
   393    395     } else {
   394    396       do_isspace_test 6.$T.19 $T   {32 160 5760 8192}

Changes to test/join.test.

   758    758   } {2 2 1 |}
   759    759   do_execsql_test join-14.11 {
   760    760     SELECT *, '|' FROM t3 LEFT JOIN v2 ON a=x WHERE b+1=x;
   761    761   } {2 2 1 |}
   762    762   do_execsql_test join-14.12 {
   763    763     SELECT *, '|' FROM t3 LEFT JOIN v2 ON a=x ORDER BY b;
   764    764   } {4 {} {} | 2 2 1 |}
          765  +
          766  +# Verify the fix for ticket
          767  +# https://www.sqlite.org/src/info/892fc34f173e99d8
          768  +#
          769  +db close
          770  +sqlite3 db :memory:
          771  +do_execsql_test join-14.20 {
          772  +  CREATE TABLE t1(id INTEGER PRIMARY KEY);
          773  +  CREATE TABLE t2(id INTEGER PRIMARY KEY, c2 INTEGER);
          774  +  CREATE TABLE t3(id INTEGER PRIMARY KEY, c3 INTEGER);
          775  +  INSERT INTO t1(id) VALUES(456);
          776  +  INSERT INTO t3(id) VALUES(1),(2);
          777  +  SELECT t1.id, x2.id, x3.id
          778  +  FROM t1
          779  +  LEFT JOIN (SELECT * FROM t2) AS x2 ON t1.id=x2.c2
          780  +  LEFT JOIN t3 AS x3 ON x2.id=x3.c3;
          781  +} {456 {} {}}
   765    782   
   766    783   finish_test

Changes to test/kvtest.c.

    67     67   "\n"
    68     68   "        Generate a new test database file named DBFILE containing N\n"
    69     69   "        BLOBs each of size M bytes.  The page size of the new database\n"
    70     70   "        file will be X.  Additional options:\n"
    71     71   "\n"
    72     72   "           --variance V           Randomly vary M by plus or minus V\n"
    73     73   "\n"
    74         -"   kvtest export DBFILE DIRECTORY\n"
           74  +"   kvtest export DBFILE DIRECTORY [--tree]\n"
    75     75   "\n"
    76     76   "        Export all the blobs in the kv table of DBFILE into separate\n"
    77         -"        files in DIRECTORY.\n"
           77  +"        files in DIRECTORY.  DIRECTORY is created if it does not previously\n"
           78  +"        exist.  If the --tree option is used, then the blobs are written\n"
           79  +"        into a hierarchy of directories, using names like 00/00/00,\n"
           80  +"        00/00/01, 00/00/02, and so forth.  Without the --tree option, all\n"
           81  +"        files are in the top-level directory with names like 000000, 000001,\n"
           82  +"        000002, and so forth.\n"
           83  +"\n"
           84  +"   kvtest stat DBFILE [options]\n"
    78     85   "\n"
    79         -"   kvtest stat DBFILE\n"
           86  +"        Display summary information about DBFILE.  Options:\n"
    80     87   "\n"
    81         -"        Display summary information about DBFILE\n"
           88  +"           --vacuum               Run VACUUM on the database file\n"
    82     89   "\n"
    83     90   "   kvtest run DBFILE [options]\n"
    84     91   "\n"
    85     92   "        Run a performance test.  DBFILE can be either the name of a\n"
    86     93   "        database or a directory containing sample files.  Options:\n"
    87     94   "\n"
    88     95   "           --asc                  Read blobs in ascending order\n"
    89     96   "           --blob-api             Use the BLOB API\n"
    90     97   "           --cache-size N         Database cache size\n"
    91     98   "           --count N              Read N blobs\n"
    92     99   "           --desc                 Read blobs in descending order\n"
          100  +"           --fsync                Synchronous file writes\n"
          101  +"           --integrity-check      Run \"PRAGMA integrity_check\" after test\n"
    93    102   "           --max-id N             Maximum blob key to use\n"
    94    103   "           --mmap N               Mmap as much as N bytes of DBFILE\n"
          104  +"           --multitrans           Each read or write in its own transaction\n"
          105  +"           --nocheckpoint         Omit the checkpoint on WAL mode writes\n"
          106  +"           --nosync               Set \"PRAGMA synchronous=OFF\"\n"
    95    107   "           --jmode MODE           Set MODE journal mode prior to starting\n"
    96    108   "           --random               Read blobs in a random order\n"
    97    109   "           --start N              Start reading with this blob key\n"
    98    110   "           --stats                Output operating stats before exiting\n"
          111  +"           --update               Do an overwrite test\n"
    99    112   ;
   100    113   
   101    114   /* Reference resources used */
   102    115   #include <stdio.h>
   103    116   #include <stdlib.h>
   104    117   #include <sys/types.h>
   105    118   #include <sys/stat.h>
................................................................................
   107    120   #include <string.h>
   108    121   #include "sqlite3.h"
   109    122   
   110    123   #ifndef _WIN32
   111    124   # include <unistd.h>
   112    125   #else
   113    126     /* Provide Windows equivalent for the needed parts of unistd.h */
          127  +# include <direct.h>
   114    128   # include <io.h>
   115    129   # define R_OK 2
   116    130   # define S_ISREG(m) (((m) & S_IFMT) == S_IFREG)
   117    131   # define S_ISDIR(m) (((m) & S_IFMT) == S_IFDIR)
   118    132   # define access _access
   119    133   #endif
   120    134   
          135  +#include <stdint.h>
          136  +
          137  +/*
          138  +** The following macros are used to cast pointers to integers and
          139  +** integers to pointers.  The way you do this varies from one compiler
          140  +** to the next, so we have developed the following set of #if statements
          141  +** to generate appropriate macros for a wide range of compilers.
          142  +**
          143  +** The correct "ANSI" way to do this is to use the intptr_t type.
          144  +** Unfortunately, that typedef is not available on all compilers, or
          145  +** if it is available, it requires an #include of specific headers
          146  +** that vary from one machine to the next.
          147  +**
          148  +** Ticket #3860:  The llvm-gcc-4.2 compiler from Apple chokes on
          149  +** the ((void*)&((char*)0)[X]) construct.  But MSVC chokes on ((void*)(X)).
          150  +** So we have to define the macros in different ways depending on the
          151  +** compiler.
          152  +*/
          153  +#if defined(__PTRDIFF_TYPE__)  /* This case should work for GCC */
          154  +# define SQLITE_INT_TO_PTR(X)  ((void*)(__PTRDIFF_TYPE__)(X))
          155  +# define SQLITE_PTR_TO_INT(X)  ((sqlite3_int64)(__PTRDIFF_TYPE__)(X))
          156  +#else
          157  +# define SQLITE_INT_TO_PTR(X)  ((void*)(intptr_t)(X))
          158  +# define SQLITE_PTR_TO_INT(X)  ((sqlite3_int64)(intptr_t)(X))
          159  +#endif
   121    160   
   122    161   /*
   123    162   ** Show thqe help text and quit.
   124    163   */
   125    164   static void showHelp(void){
   126    165     fprintf(stdout, "%s", zHelp);
   127    166     exit(1);
................................................................................
   197    236     return isNeg? -v : v;
   198    237   }
   199    238   
   200    239   
   201    240   /*
   202    241   ** Check the filesystem object zPath.  Determine what it is:
   203    242   **
   204         -**    PATH_DIR     A directory
          243  +**    PATH_DIR     A single directory holding many files
          244  +**    PATH_TREE    A directory hierarchy with files at the leaves
   205    245   **    PATH_DB      An SQLite database
   206    246   **    PATH_NEXIST  Does not exist
   207    247   **    PATH_OTHER   Something else
          248  +**
          249  +** PATH_DIR means all of the separate files are grouped together
          250  +** into a single directory with names like 000000, 000001, 000002, and
          251  +** so forth.  PATH_TREE means there is a hierarchy of directories so
          252  +** that no single directory has too many entries.  The files have names
          253  +** like 00/00/00, 00/00/01, 00/00/02 and so forth.  The decision between
          254  +** PATH_DIR and PATH_TREE is determined by the presence of a subdirectory
          255  +** named "00" at the top-level.
   208    256   */
   209    257   #define PATH_DIR     1
   210         -#define PATH_DB      2
          258  +#define PATH_TREE    2
          259  +#define PATH_DB      3
   211    260   #define PATH_NEXIST  0
   212    261   #define PATH_OTHER   99
   213    262   static int pathType(const char *zPath){
   214    263     struct stat x;
   215    264     int rc;
   216    265     if( access(zPath,R_OK) ) return PATH_NEXIST;
   217    266     memset(&x, 0, sizeof(x));
   218    267     rc = stat(zPath, &x);
   219    268     if( rc<0 ) return PATH_OTHER;
   220         -  if( S_ISDIR(x.st_mode) ) return PATH_DIR;
          269  +  if( S_ISDIR(x.st_mode) ){
          270  +    char *zLayer1 = sqlite3_mprintf("%s/00", zPath);
          271  +    memset(&x, 0, sizeof(x));
          272  +    rc = stat(zLayer1, &x);
          273  +    sqlite3_free(zLayer1);
          274  +    if( rc<0 ) return PATH_DIR;
          275  +    if( S_ISDIR(x.st_mode) ) return PATH_TREE;
          276  +    return PATH_DIR;
          277  +  }
   221    278     if( (x.st_size%512)==0 ) return PATH_DB;
   222    279     return PATH_OTHER;
   223    280   }
   224    281   
   225    282   /*
   226    283   ** Return the size of a file in bytes.  Or return -1 if the
   227    284   ** named object is not a regular file or does not exist.
................................................................................
   324    381   */
   325    382   static int statMain(int argc, char **argv){
   326    383     char *zDb;
   327    384     int i, rc;
   328    385     sqlite3 *db;
   329    386     char *zSql;
   330    387     sqlite3_stmt *pStmt;
          388  +  int doVacuum = 0;
   331    389   
   332    390     assert( strcmp(argv[1],"stat")==0 );
   333    391     assert( argc>=3 );
   334    392     zDb = argv[2];
   335    393     for(i=3; i<argc; i++){
   336    394       char *z = argv[i];
   337    395       if( z[0]!='-' ) fatalError("unknown argument: \"%s\"", z);
   338    396       if( z[1]=='-' ) z++;
          397  +    if( strcmp(z, "-vacuum")==0 ){
          398  +      doVacuum = 1;
          399  +      continue;
          400  +    }
   339    401       fatalError("unknown option: \"%s\"", argv[i]);
   340    402     }
   341    403     rc = sqlite3_open(zDb, &db);
   342    404     if( rc ){
   343    405       fatalError("cannot open database \"%s\": %s", zDb, sqlite3_errmsg(db));
   344    406     }
          407  +  if( doVacuum ){
          408  +    printf("Vacuuming...."); fflush(stdout);
          409  +    sqlite3_exec(db, "VACUUM", 0, 0, 0);
          410  +    printf("       done\n");
          411  +  }
   345    412     zSql = sqlite3_mprintf(
   346    413       "SELECT count(*), min(length(v)), max(length(v)), avg(length(v))"
   347    414       "  FROM kv"
   348    415     );
   349    416     rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
   350    417     if( rc ) fatalError("cannot prepare SQL [%s]: %s", zSql, sqlite3_errmsg(db));
   351    418     sqlite3_free(zSql);
................................................................................
   369    436     zSql = sqlite3_mprintf("PRAGMA page_count");
   370    437     rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
   371    438     if( rc ) fatalError("cannot prepare SQL [%s]: %s", zSql, sqlite3_errmsg(db));
   372    439     sqlite3_free(zSql);
   373    440     if( sqlite3_step(pStmt)==SQLITE_ROW ){
   374    441       printf("Page-count:         %8d\n", sqlite3_column_int(pStmt, 0));
   375    442     }
          443  +  sqlite3_finalize(pStmt);
          444  +  zSql = sqlite3_mprintf("PRAGMA freelist_count");
          445  +  rc = sqlite3_prepare_v2(db, zSql, -1, &pStmt, 0);
          446  +  if( rc ) fatalError("cannot prepare SQL [%s]: %s", zSql, sqlite3_errmsg(db));
          447  +  sqlite3_free(zSql);
          448  +  if( sqlite3_step(pStmt)==SQLITE_ROW ){
          449  +    printf("Freelist-count:     %8d\n", sqlite3_column_int(pStmt, 0));
          450  +  }
          451  +  sqlite3_finalize(pStmt);
          452  +  rc = sqlite3_prepare_v2(db, "PRAGMA integrity_check(10)", -1, &pStmt, 0);
          453  +  if( rc ) fatalError("cannot prepare integrity check: %s", sqlite3_errmsg(db));
          454  +  while( sqlite3_step(pStmt)==SQLITE_ROW ){
          455  +    printf("Integrity-check:    %s\n", sqlite3_column_text(pStmt, 0));
          456  +  }
   376    457     sqlite3_finalize(pStmt);
   377    458     sqlite3_close(db);
   378    459     return 0;
   379    460   }
   380    461   
   381    462   /*
   382         -** Implementation of the "writefile(X,Y)" SQL function.  The argument Y
   383         -** is written into file X.  The number of bytes written is returned.  Or
   384         -** NULL is returned if something goes wrong, such as being unable to open
   385         -** file X for writing.
          463  +**      remember(V,PTR)
          464  +**
          465  +** Return the integer value V.  Also save the value of V in a
          466  +** C-language variable whose address is PTR.
   386    467   */
   387         -static void writefileFunc(
   388         -  sqlite3_context *context,
          468  +static void rememberFunc(
          469  +  sqlite3_context *pCtx,
   389    470     int argc,
   390    471     sqlite3_value **argv
   391    472   ){
   392         -  FILE *out;
   393         -  const char *z;
   394         -  sqlite3_int64 rc;
   395         -  const char *zFile;
          473  +  sqlite3_int64 v;
          474  +  sqlite3_int64 ptr;
          475  +  assert( argc==2 );
          476  +  v = sqlite3_value_int64(argv[0]);
          477  +  ptr = sqlite3_value_int64(argv[1]);
          478  +  *(sqlite3_int64*)SQLITE_INT_TO_PTR(ptr) = v;
          479  +  sqlite3_result_int64(pCtx, v);
          480  +}
   396    481   
   397         -  zFile = (const char*)sqlite3_value_text(argv[0]);
   398         -  if( zFile==0 ) return;
   399         -  out = fopen(zFile, "wb");
   400         -  if( out==0 ) return;
   401         -  z = (const char*)sqlite3_value_blob(argv[1]);
   402         -  if( z==0 ){
   403         -    rc = 0;
   404         -  }else{
   405         -    rc = fwrite(z, 1, sqlite3_value_bytes(argv[1]), out);
   406         -  }
   407         -  fclose(out);
   408         -  printf("\r%s   ", zFile); fflush(stdout);
   409         -  sqlite3_result_int64(context, rc);
          482  +/*
          483  +** Make sure a directory named zDir exists.
          484  +*/
          485  +static void kvtest_mkdir(const char *zDir){
          486  +#if defined(_WIN32)
          487  +  (void)mkdir(zDir);
          488  +#else
          489  +  (void)mkdir(zDir, 0755);
          490  +#endif
   410    491   }
   411    492   
   412    493   /*
   413    494   ** Export the kv table to individual files in the filesystem
   414    495   */
   415    496   static int exportMain(int argc, char **argv){
   416    497     char *zDb;
   417    498     char *zDir;
   418    499     sqlite3 *db;
   419         -  char *zSql;
          500  +  sqlite3_stmt *pStmt;
   420    501     int rc;
   421         -  char *zErrMsg = 0;
          502  +  int ePathType;
          503  +  int nFN;
          504  +  char *zFN;
          505  +  char *zTail;
          506  +  size_t nWrote;
          507  +  int i;
   422    508   
   423    509     assert( strcmp(argv[1],"export")==0 );
   424    510     assert( argc>=3 );
          511  +  if( argc<4 ) fatalError("Usage: kvtest export DATABASE DIRECTORY [OPTIONS]");
   425    512     zDb = argv[2];
   426         -  if( argc!=4 ) fatalError("Usage: kvtest export DATABASE DIRECTORY");
   427    513     zDir = argv[3];
   428         -  if( pathType(zDir)!=PATH_DIR ){
          514  +  kvtest_mkdir(zDir);
          515  +  for(i=4; i<argc; i++){
          516  +    const char *z = argv[i];
          517  +    if( z[0]=='-' && z[1]=='-' ) z++;
          518  +    if( strcmp(z,"-tree")==0 ){
          519  +      zFN = sqlite3_mprintf("%s/00", zDir);
          520  +      kvtest_mkdir(zFN);
          521  +      sqlite3_free(zFN);
          522  +      continue;
          523  +    }
          524  +    fatalError("unknown argument: \"%s\"\n", argv[i]);
          525  +  }
          526  +  ePathType = pathType(zDir);
          527  +  if( ePathType!=PATH_DIR && ePathType!=PATH_TREE ){
   429    528       fatalError("object \"%s\" is not a directory", zDir);
   430    529     }
   431    530     rc = sqlite3_open(zDb, &db);
   432    531     if( rc ){
   433    532       fatalError("cannot open database \"%s\": %s", zDb, sqlite3_errmsg(db));
   434    533     }
   435         -  sqlite3_create_function(db, "writefile", 2, SQLITE_UTF8, 0,
   436         -                          writefileFunc, 0, 0);
   437         -  zSql = sqlite3_mprintf(
   438         -    "SELECT writefile(printf('%s/%%06d',k),v) FROM kv;",
   439         -    zDir
   440         -  );
   441         -  rc = sqlite3_exec(db, zSql, 0, 0, &zErrMsg);
   442         -  if( rc ) fatalError("database create failed: %s", zErrMsg);
   443         -  sqlite3_free(zSql);
          534  +  rc = sqlite3_prepare_v2(db, "SELECT k, v FROM kv ORDER BY k", -1, &pStmt, 0);
          535  +  if( rc ){
          536  +    fatalError("prepare_v2 failed: %s\n", sqlite3_errmsg(db));
          537  +  }
          538  +  nFN = (int)strlen(zDir);
          539  +  zFN = sqlite3_mprintf("%s/00/00/00.extra---------------------", zDir);
          540  +  if( zFN==0 ){
          541  +    fatalError("malloc failed\n");
          542  +  }
          543  +  zTail = zFN + nFN + 1;
          544  +  while( sqlite3_step(pStmt)==SQLITE_ROW ){
          545  +    int iKey = sqlite3_column_int(pStmt, 0);
          546  +    sqlite3_int64 nData = sqlite3_column_bytes(pStmt, 1);
          547  +    const void *pData = sqlite3_column_blob(pStmt, 1);
          548  +    FILE *out;
          549  +    if( ePathType==PATH_DIR ){
          550  +      sqlite3_snprintf(20, zTail, "%06d", iKey);
          551  +    }else{
          552  +      sqlite3_snprintf(20, zTail, "%02d", iKey/10000);
          553  +      kvtest_mkdir(zFN);
          554  +      sqlite3_snprintf(20, zTail, "%02d/%02d", iKey/10000, (iKey/100)%100);
          555  +      kvtest_mkdir(zFN);
          556  +      sqlite3_snprintf(20, zTail, "%02d/%02d/%02d",
          557  +                       iKey/10000, (iKey/100)%100, iKey%100);
          558  +    }
          559  +    out = fopen(zFN, "wb");      
          560  +    nWrote = fwrite(pData, 1, nData, out);
          561  +    fclose(out);
          562  +    printf("\r%s   ", zTail); fflush(stdout);
          563  +    if( nWrote!=nData ){
          564  +      fatalError("Wrote only %d of %d bytes to %s\n",
          565  +                  (int)nWrote, nData, zFN);
          566  +    }
          567  +  }
          568  +  sqlite3_finalize(pStmt);
   444    569     sqlite3_close(db);
          570  +  sqlite3_free(zFN);
   445    571     printf("\n");
   446    572     return 0;
   447    573   }
   448    574   
   449    575   /*
   450    576   ** Read the content of file zName into memory obtained from sqlite3_malloc64()
   451    577   ** and return a pointer to the buffer. The caller is responsible for freeing 
................................................................................
   457    583   ** For convenience, a nul-terminator byte is always appended to the data read
   458    584   ** from the file before the buffer is returned. This byte is not included in
   459    585   ** the final value of (*pnByte), if applicable.
   460    586   **
   461    587   ** NULL is returned if any error is encountered. The final value of *pnByte
   462    588   ** is undefined in this case.
   463    589   */
   464         -static unsigned char *readFile(const char *zName, int *pnByte){
          590  +static unsigned char *readFile(const char *zName, sqlite3_int64 *pnByte){
   465    591     FILE *in;               /* FILE from which to read content of zName */
   466    592     sqlite3_int64 nIn;      /* Size of zName in bytes */
   467    593     size_t nRead;           /* Number of bytes actually read */
   468    594     unsigned char *pBuf;    /* Content read from disk */
   469    595   
   470    596     nIn = fileSize(zName);
   471    597     if( nIn<0 ) return 0;
................................................................................
   475    601     if( pBuf==0 ) return 0;
   476    602     nRead = fread(pBuf, (size_t)nIn, 1, in);
   477    603     fclose(in);
   478    604     if( nRead!=1 ){
   479    605       sqlite3_free(pBuf);
   480    606       return 0;
   481    607     }
   482         -  if( pnByte ) *pnByte = (int)nIn;
          608  +  if( pnByte ) *pnByte = nIn;
   483    609     return pBuf;
   484    610   }
          611  +
          612  +/*
          613  +** Overwrite a file with randomness.  Do not change the size of the
          614  +** file.
          615  +*/
          616  +static void updateFile(const char *zName, sqlite3_int64 *pnByte, int doFsync){
          617  +  FILE *out;              /* FILE from which to read content of zName */
          618  +  sqlite3_int64 sz;       /* Size of zName in bytes */
          619  +  size_t nWritten;        /* Number of bytes actually read */
          620  +  unsigned char *pBuf;    /* Content to store on disk */
          621  +  const char *zMode = "wb";   /* Mode for fopen() */
          622  +
          623  +  sz = fileSize(zName);
          624  +  if( sz<0 ){
          625  +    fatalError("No such file: \"%s\"", zName);
          626  +  }
          627  +  *pnByte = sz;
          628  +  if( sz==0 ) return;
          629  +  pBuf = sqlite3_malloc64( sz );
          630  +  if( pBuf==0 ){
          631  +    fatalError("Cannot allocate %lld bytes\n", sz);
          632  +  }
          633  +  sqlite3_randomness((int)sz, pBuf); 
          634  +#if defined(_WIN32)
          635  +  if( doFsync ) zMode = "wbc";
          636  +#endif
          637  +  out = fopen(zName, zMode);
          638  +  if( out==0 ){
          639  +    fatalError("Cannot open \"%s\" for writing\n", zName);
          640  +  }
          641  +  nWritten = fwrite(pBuf, 1, (size_t)sz, out);
          642  +  if( doFsync ){
          643  +#if defined(_WIN32)
          644  +    fflush(out);
          645  +#else
          646  +    fsync(fileno(out));
          647  +#endif
          648  +  }
          649  +  fclose(out);
          650  +  if( nWritten!=(size_t)sz ){
          651  +    fatalError("Wrote only %d of %d bytes to \"%s\"\n",
          652  +               (int)nWritten, (int)sz, zName);
          653  +  }
          654  +  sqlite3_free(pBuf);
          655  +}
   485    656   
   486    657   /*
   487    658   ** Return the current time in milliseconds since the beginning of
   488    659   ** the Julian epoch.
   489    660   */
   490    661   static sqlite3_int64 timeOfDay(void){
   491    662     static sqlite3_vfs *clockVfs = 0;
................................................................................
   633    804     int iKey = 1;               /* Next blob key */
   634    805     int iMax = 0;               /* Largest allowed key */
   635    806     int iPagesize = 0;          /* Database page size */
   636    807     int iCache = 1000;          /* Database cache size in kibibytes */
   637    808     int bBlobApi = 0;           /* Use the incremental blob I/O API */
   638    809     int bStats = 0;             /* Print stats before exiting */
   639    810     int eOrder = ORDER_ASC;     /* Access order */
          811  +  int isUpdateTest = 0;       /* Do in-place updates rather than reads */
          812  +  int doIntegrityCk = 0;      /* Run PRAGMA integrity_check after the test */
          813  +  int noSync = 0;             /* Disable synchronous mode */
          814  +  int doFsync = 0;            /* Update disk files synchronously */
          815  +  int doMultiTrans = 0;       /* Each operation in its own transaction */
          816  +  int noCheckpoint = 0;       /* Omit the checkpoint in WAL mode */
   640    817     sqlite3 *db = 0;            /* Database connection */
   641    818     sqlite3_stmt *pStmt = 0;    /* Prepared statement for SQL access */
   642    819     sqlite3_blob *pBlob = 0;    /* Handle for incremental Blob I/O */
   643    820     sqlite3_int64 tmStart;      /* Start time */
   644    821     sqlite3_int64 tmElapsed;    /* Elapsed time */
   645    822     int mmapSize = 0;           /* --mmap N argument */
   646         -  int nData = 0;              /* Bytes of data */
          823  +  sqlite3_int64 nData = 0;    /* Bytes of data */
   647    824     sqlite3_int64 nTotal = 0;   /* Total data read */
   648    825     unsigned char *pData = 0;   /* Content of the blob */
   649         -  int nAlloc = 0;             /* Space allocated for pData[] */
          826  +  sqlite3_int64 nAlloc = 0;   /* Space allocated for pData[] */
   650    827     const char *zJMode = 0;     /* Journal mode */
   651    828     
   652    829   
   653    830     assert( strcmp(argv[1],"run")==0 );
   654    831     assert( argc>=3 );
   655    832     zDb = argv[2];
   656    833     eType = pathType(zDb);
   657    834     if( eType==PATH_OTHER ) fatalError("unknown object type: \"%s\"", zDb);
   658    835     if( eType==PATH_NEXIST ) fatalError("object does not exist: \"%s\"", zDb);
   659    836     for(i=3; i<argc; i++){
   660    837       char *z = argv[i];
   661    838       if( z[0]!='-' ) fatalError("unknown argument: \"%s\"", z);
   662    839       if( z[1]=='-' ) z++;
          840  +    if( strcmp(z, "-asc")==0 ){
          841  +      eOrder = ORDER_ASC;
          842  +      continue;
          843  +    }
          844  +    if( strcmp(z, "-blob-api")==0 ){
          845  +      bBlobApi = 1;
          846  +      continue;
          847  +    }
          848  +    if( strcmp(z, "-cache-size")==0 ){
          849  +      if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
          850  +      iCache = integerValue(argv[++i]);
          851  +      continue;
          852  +    }
   663    853       if( strcmp(z, "-count")==0 ){
   664    854         if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
   665    855         nCount = integerValue(argv[++i]);
   666    856         if( nCount<1 ) fatalError("the --count must be positive");
   667    857         continue;
          858  +    }
          859  +    if( strcmp(z, "-desc")==0 ){
          860  +      eOrder = ORDER_DESC;
          861  +      continue;
          862  +    }
          863  +    if( strcmp(z, "-fsync")==0 ){
          864  +      doFsync = 1;
          865  +      continue;
          866  +    }
          867  +    if( strcmp(z, "-integrity-check")==0 ){
          868  +      doIntegrityCk = 1;
          869  +      continue;
          870  +    }
          871  +    if( strcmp(z, "-jmode")==0 ){
          872  +      if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
          873  +      zJMode = argv[++i];
          874  +      continue;
   668    875       }
   669    876       if( strcmp(z, "-mmap")==0 ){
   670    877         if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
   671    878         mmapSize = integerValue(argv[++i]);
   672    879         if( nCount<0 ) fatalError("the --mmap must be non-negative");
   673    880         continue;
   674    881       }
   675    882       if( strcmp(z, "-max-id")==0 ){
   676    883         if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
   677    884         iMax = integerValue(argv[++i]);
   678    885         continue;
   679    886       }
   680         -    if( strcmp(z, "-start")==0 ){
   681         -      if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
   682         -      iKey = integerValue(argv[++i]);
   683         -      if( iKey<1 ) fatalError("the --start must be positive");
          887  +    if( strcmp(z, "-multitrans")==0 ){
          888  +      doMultiTrans = 1;
          889  +      continue;
          890  +    }
          891  +    if( strcmp(z, "-nocheckpoint")==0 ){
          892  +      noCheckpoint = 1;
   684    893         continue;
   685    894       }
   686         -    if( strcmp(z, "-cache-size")==0 ){
   687         -      if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
   688         -      iCache = integerValue(argv[++i]);
   689         -      continue;
   690         -    }
   691         -    if( strcmp(z, "-jmode")==0 ){
   692         -      if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
   693         -      zJMode = argv[++i];
          895  +    if( strcmp(z, "-nosync")==0 ){
          896  +      noSync = 1;
   694    897         continue;
   695    898       }
   696    899       if( strcmp(z, "-random")==0 ){
   697    900         eOrder = ORDER_RANDOM;
   698    901         continue;
   699    902       }
   700         -    if( strcmp(z, "-asc")==0 ){
   701         -      eOrder = ORDER_ASC;
   702         -      continue;
   703         -    }
   704         -    if( strcmp(z, "-desc")==0 ){
   705         -      eOrder = ORDER_DESC;
   706         -      continue;
   707         -    }
   708         -    if( strcmp(z, "-blob-api")==0 ){
   709         -      bBlobApi = 1;
          903  +    if( strcmp(z, "-start")==0 ){
          904  +      if( i==argc-1 ) fatalError("missing argument on \"%s\"", argv[i]);
          905  +      iKey = integerValue(argv[++i]);
          906  +      if( iKey<1 ) fatalError("the --start must be positive");
   710    907         continue;
   711    908       }
   712    909       if( strcmp(z, "-stats")==0 ){
   713    910         bStats = 1;
   714    911         continue;
          912  +    }
          913  +    if( strcmp(z, "-update")==0 ){
          914  +      isUpdateTest = 1;
          915  +      continue;
   715    916       }
   716    917       fatalError("unknown option: \"%s\"", argv[i]);
          918  +  }
          919  +  if( eType==PATH_DB ){
          920  +    /* Recover any prior crashes prior to starting the timer */
          921  +    sqlite3_open(zDb, &db);
          922  +    sqlite3_exec(db, "SELECT rowid FROM sqlite_master LIMIT 1", 0, 0, 0);
          923  +    sqlite3_close(db);
          924  +    db = 0;
   717    925     }
   718    926     tmStart = timeOfDay();
   719    927     if( eType==PATH_DB ){
   720    928       char *zSql;
   721    929       rc = sqlite3_open(zDb, &db);
   722    930       if( rc ){
   723    931         fatalError("cannot open database \"%s\": %s", zDb, sqlite3_errmsg(db));
   724    932       }
   725    933       zSql = sqlite3_mprintf("PRAGMA mmap_size=%d", mmapSize);
   726    934       sqlite3_exec(db, zSql, 0, 0, 0);
          935  +    sqlite3_free(zSql);
   727    936       zSql = sqlite3_mprintf("PRAGMA cache_size=%d", iCache);
   728    937       sqlite3_exec(db, zSql, 0, 0, 0);
   729    938       sqlite3_free(zSql);
          939  +    if( noSync ){
          940  +      sqlite3_exec(db, "PRAGMA synchronous=OFF", 0, 0, 0);
          941  +    }
   730    942       pStmt = 0;
   731    943       sqlite3_prepare_v2(db, "PRAGMA page_size", -1, &pStmt, 0);
   732    944       if( sqlite3_step(pStmt)==SQLITE_ROW ){
   733    945         iPagesize = sqlite3_column_int(pStmt, 0);
   734    946       }
   735    947       sqlite3_finalize(pStmt);
   736    948       sqlite3_prepare_v2(db, "PRAGMA cache_size", -1, &pStmt, 0);
................................................................................
   741    953       }
   742    954       sqlite3_finalize(pStmt);
   743    955       pStmt = 0;
   744    956       if( zJMode ){
   745    957         zSql = sqlite3_mprintf("PRAGMA journal_mode=%Q", zJMode);
   746    958         sqlite3_exec(db, zSql, 0, 0, 0);
   747    959         sqlite3_free(zSql);
          960  +      if( noCheckpoint ){
          961  +        sqlite3_exec(db, "PRAGMA wal_autocheckpoint=0", 0, 0, 0);
          962  +      }
   748    963       }
   749    964       sqlite3_prepare_v2(db, "PRAGMA journal_mode", -1, &pStmt, 0);
   750    965       if( sqlite3_step(pStmt)==SQLITE_ROW ){
   751    966         zJMode = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 0));
   752    967       }else{
   753    968         zJMode = "???";
   754    969       }
................................................................................
   757    972         sqlite3_prepare_v2(db, "SELECT max(k) FROM kv", -1, &pStmt, 0);
   758    973         if( sqlite3_step(pStmt)==SQLITE_ROW ){
   759    974           iMax = sqlite3_column_int(pStmt, 0);
   760    975         }
   761    976         sqlite3_finalize(pStmt);
   762    977       }
   763    978       pStmt = 0;
   764         -    sqlite3_exec(db, "BEGIN", 0, 0, 0);
          979  +    if( !doMultiTrans ) sqlite3_exec(db, "BEGIN", 0, 0, 0);
   765    980     }
   766    981     if( iMax<=0 ) iMax = 1000;
   767    982     for(i=0; i<nCount; i++){
   768         -    if( eType==PATH_DIR ){
   769         -      /* CASE 1: Reading blobs out of separate files */
          983  +    if( eType==PATH_DIR || eType==PATH_TREE ){
          984  +      /* CASE 1: Reading or writing blobs out of separate files */
   770    985         char *zKey;
   771         -      zKey = sqlite3_mprintf("%s/%06d", zDb, iKey);
          986  +      if( eType==PATH_DIR ){
          987  +        zKey = sqlite3_mprintf("%s/%06d", zDb, iKey);
          988  +      }else{
          989  +        zKey = sqlite3_mprintf("%s/%02d/%02d/%02d", zDb, iKey/10000,
          990  +                               (iKey/100)%100, iKey%100);
          991  +      }
   772    992         nData = 0;
   773         -      pData = readFile(zKey, &nData);
          993  +      if( isUpdateTest ){
          994  +        updateFile(zKey, &nData, doFsync);
          995  +      }else{
          996  +        pData = readFile(zKey, &nData);
          997  +        sqlite3_free(pData);
          998  +      }
   774    999         sqlite3_free(zKey);
   775         -      sqlite3_free(pData);
   776   1000       }else if( bBlobApi ){
   777   1001         /* CASE 2: Reading from database using the incremental BLOB I/O API */
   778   1002         if( pBlob==0 ){
   779         -        rc = sqlite3_blob_open(db, "main", "kv", "v", iKey, 0, &pBlob);
         1003  +        rc = sqlite3_blob_open(db, "main", "kv", "v", iKey,
         1004  +                               isUpdateTest, &pBlob);
   780   1005           if( rc ){
   781   1006             fatalError("could not open sqlite3_blob handle: %s",
   782   1007                        sqlite3_errmsg(db));
   783   1008           }
   784   1009         }else{
   785   1010           rc = sqlite3_blob_reopen(pBlob, iKey);
   786   1011         }
   787   1012         if( rc==SQLITE_OK ){
   788   1013           nData = sqlite3_blob_bytes(pBlob);
   789   1014           if( nAlloc<nData+1 ){
   790   1015             nAlloc = nData+100;
   791         -          pData = sqlite3_realloc(pData, nAlloc);
         1016  +          pData = sqlite3_realloc64(pData, nAlloc);
   792   1017           }
   793   1018           if( pData==0 ) fatalError("cannot allocate %d bytes", nData+1);
   794         -        rc = sqlite3_blob_read(pBlob, pData, nData, 0);
   795         -        if( rc!=SQLITE_OK ){
   796         -          fatalError("could not read the blob at %d: %s", iKey,
   797         -                     sqlite3_errmsg(db));
         1019  +        if( isUpdateTest ){
         1020  +          sqlite3_randomness((int)nData, pData);
         1021  +          rc = sqlite3_blob_write(pBlob, pData, (int)nData, 0);
         1022  +          if( rc!=SQLITE_OK ){
         1023  +            fatalError("could not write the blob at %d: %s", iKey,
         1024  +                      sqlite3_errmsg(db));
         1025  +          }
         1026  +        }else{
         1027  +          rc = sqlite3_blob_read(pBlob, pData, (int)nData, 0);
         1028  +          if( rc!=SQLITE_OK ){
         1029  +            fatalError("could not read the blob at %d: %s", iKey,
         1030  +                      sqlite3_errmsg(db));
         1031  +          }
   798   1032           }
   799   1033         }
   800   1034       }else{
   801   1035         /* CASE 3: Reading from database using SQL */
   802   1036         if( pStmt==0 ){
   803         -        rc = sqlite3_prepare_v2(db, 
   804         -               "SELECT v FROM kv WHERE k=?1", -1, &pStmt, 0);
         1037  +        if( isUpdateTest ){
         1038  +          sqlite3_create_function(db, "remember", 2, SQLITE_UTF8, 0,
         1039  +                                  rememberFunc, 0, 0);
         1040  +
         1041  +          rc = sqlite3_prepare_v2(db, 
         1042  +            "UPDATE kv SET v=randomblob(remember(length(v),?2))"
         1043  +            " WHERE k=?1", -1, &pStmt, 0);
         1044  +          sqlite3_bind_int64(pStmt, 2, SQLITE_PTR_TO_INT(&nData));
         1045  +        }else{
         1046  +          rc = sqlite3_prepare_v2(db, 
         1047  +                 "SELECT v FROM kv WHERE k=?1", -1, &pStmt, 0);
         1048  +        }
   805   1049           if( rc ){
   806   1050             fatalError("cannot prepare query: %s", sqlite3_errmsg(db));
   807   1051           }
   808   1052         }else{
   809   1053           sqlite3_reset(pStmt);
   810   1054         }
   811   1055         sqlite3_bind_int(pStmt, 1, iKey);
         1056  +      nData = 0;
   812   1057         rc = sqlite3_step(pStmt);
   813   1058         if( rc==SQLITE_ROW ){
   814   1059           nData = sqlite3_column_bytes(pStmt, 0);
   815   1060           pData = (unsigned char*)sqlite3_column_blob(pStmt, 0);
   816         -      }else{
   817         -        nData = 0;
   818   1061         }
   819   1062       }
   820   1063       if( eOrder==ORDER_ASC ){
   821   1064         iKey++;
   822   1065         if( iKey>iMax ) iKey = 1;
   823   1066       }else if( eOrder==ORDER_DESC ){
   824   1067         iKey--;
................................................................................
   831   1074     }
   832   1075     if( nAlloc ) sqlite3_free(pData);
   833   1076     if( pStmt ) sqlite3_finalize(pStmt);
   834   1077     if( pBlob ) sqlite3_blob_close(pBlob);
   835   1078     if( bStats ){
   836   1079       display_stats(db, 0);
   837   1080     }
   838         -  if( db ) sqlite3_close(db);
         1081  +  if( db ){
         1082  +    if( !doMultiTrans ) sqlite3_exec(db, "COMMIT", 0, 0, 0);
         1083  +    if( !noCheckpoint ){
         1084  +      sqlite3_close(db);
         1085  +      db = 0;
         1086  +    }
         1087  +  }
   839   1088     tmElapsed = timeOfDay() - tmStart;
         1089  +  if( db && noCheckpoint ){
         1090  +    sqlite3_close(db);
         1091  +    db = 0;
         1092  +  }
   840   1093     if( nExtra ){
   841   1094       printf("%d cycles due to %d misses\n", nCount, nExtra);
   842   1095     }
   843   1096     if( eType==PATH_DB ){
   844   1097       printf("SQLite version: %s\n", sqlite3_libversion());
         1098  +    if( doIntegrityCk ){
         1099  +      sqlite3_open(zDb, &db);
         1100  +      sqlite3_prepare_v2(db, "PRAGMA integrity_check", -1, &pStmt, 0);
         1101  +      while( sqlite3_step(pStmt)==SQLITE_ROW ){
         1102  +        printf("integrity-check: %s\n", sqlite3_column_text(pStmt, 0));
         1103  +      }
         1104  +      sqlite3_finalize(pStmt);
         1105  +      sqlite3_close(db);
         1106  +      db = 0;
         1107  +    }
   845   1108     }
   846   1109     printf("--count %d --max-id %d", nCount-nExtra, iMax);
   847   1110     switch( eOrder ){
   848   1111       case ORDER_RANDOM:  printf(" --random\n");  break;
   849   1112       case ORDER_DESC:    printf(" --desc\n");    break;
   850   1113       default:            printf(" --asc\n");     break;
   851   1114     }
   852   1115     if( eType==PATH_DB ){
   853   1116       printf("--cache-size %d --jmode %s\n", iCache, zJMode);
   854   1117       printf("--mmap %d%s\n", mmapSize, bBlobApi ? " --blob-api" : "");
         1118  +    if( noSync ) printf("--nosync\n");
   855   1119     }
   856   1120     if( iPagesize ) printf("Database page size: %d\n", iPagesize);
   857   1121     printf("Total elapsed time: %.3f\n", tmElapsed/1000.0);
   858         -  printf("Microseconds per BLOB read: %.3f\n", tmElapsed*1000.0/nCount);
   859         -  printf("Content read rate: %.1f MB/s\n", nTotal/(1000.0*tmElapsed));
         1122  +  if( isUpdateTest ){
         1123  +    printf("Microseconds per BLOB write: %.3f\n", tmElapsed*1000.0/nCount);
         1124  +    printf("Content write rate: %.1f MB/s\n", nTotal/(1000.0*tmElapsed));
         1125  +  }else{
         1126  +    printf("Microseconds per BLOB read: %.3f\n", tmElapsed*1000.0/nCount);
         1127  +    printf("Content read rate: %.1f MB/s\n", nTotal/(1000.0*tmElapsed));
         1128  +  }
   860   1129     return 0;
   861   1130   }
   862   1131   
   863   1132   
   864   1133   int main(int argc, char **argv){
   865   1134     if( argc<3 ) showHelp();
   866   1135     if( strcmp(argv[1],"init")==0 ){

Changes to test/releasetest.tcl.

   175    175       --enable-json1 --enable-fts5 --enable-session
   176    176     }
   177    177     "Locking-Style" {
   178    178       -O2
   179    179       -DSQLITE_ENABLE_LOCKING_STYLE=1
   180    180     }
   181    181     "Apple" {
   182         -    -O1   # Avoid a compiler bug in gcc 4.2.1 build 5658
          182  +    -Os
   183    183       -DHAVE_GMTIME_R=1
   184    184       -DHAVE_ISNAN=1
   185    185       -DHAVE_LOCALTIME_R=1
   186    186       -DHAVE_PREAD=1
   187    187       -DHAVE_PWRITE=1
   188    188       -DHAVE_USLEEP=1
   189    189       -DHAVE_USLEEP=1

Changes to test/vtabH.test.

   212    212         set fd [open $path w]
   213    213         puts -nonewline $fd [string repeat 1 $sz]
   214    214         close $fd
   215    215       }
   216    216     } {}
   217    217   
   218    218     set pwd [pwd]
   219         -  do_execsql_test 3.5 {
   220         -    SELECT path, size FROM fstree WHERE path GLOB $pwd || '/subdir/*' ORDER BY 1
   221         -  } [list \
   222         -    "$pwd/subdir/x1.txt" 143 \
   223         -    "$pwd/subdir/x2.txt" 153 \
   224         -  ]
   225         -  do_execsql_test 3.6 {
   226         -    SELECT path, size FROM fstree WHERE path LIKE $pwd || '/subdir/%' ORDER BY 1
   227         -  } [list \
   228         -    "$pwd/subdir/x1.txt" 143 \
   229         -    "$pwd/subdir/x2.txt" 153 \
   230         -  ]
   231         -  do_execsql_test 3.7 {
   232         -    SELECT sum(size) FROM fstree WHERE path LIKE $pwd || '/subdir/%'
   233         -  } 296
   234         -  do_execsql_test 3.8 {
   235         -    SELECT size FROM fstree WHERE path = $pwd || '/subdir/x1.txt'
   236         -  } 143
          219  +  if {![string match {*[_%]*} $pwd]} {
          220  +    do_execsql_test 3.5 {
          221  +      SELECT path, size FROM fstree 
          222  +       WHERE path GLOB $pwd || '/subdir/*' ORDER BY 1
          223  +    } [list \
          224  +      "$pwd/subdir/x1.txt" 143 \
          225  +      "$pwd/subdir/x2.txt" 153 \
          226  +    ]
          227  +    do_execsql_test 3.6 {
          228  +      SELECT path, size FROM fstree
          229  +       WHERE path LIKE $pwd || '/subdir/%' ORDER BY 1
          230  +    } [list \
          231  +      "$pwd/subdir/x1.txt" 143 \
          232  +      "$pwd/subdir/x2.txt" 153 \
          233  +    ]
          234  +    do_execsql_test 3.7 {
          235  +      SELECT sum(size) FROM fstree WHERE path LIKE $pwd || '/subdir/%'
          236  +    } 296
          237  +    do_execsql_test 3.8 {
          238  +      SELECT size FROM fstree WHERE path = $pwd || '/subdir/x1.txt'
          239  +    } 143
          240  +  }
   237    241   
   238    242   }
   239    243   
   240    244   
   241    245   finish_test

Changes to test/whereF.test.

   114    114   
   115    115   do_execsql_test 4.0 {
   116    116     CREATE TABLE t4(a,b,c,d,e, PRIMARY KEY(a,b,c));
   117    117     CREATE INDEX t4adc ON t4(a,d,c);
   118    118     CREATE UNIQUE INDEX t4aebc ON t4(a,e,b,c);
   119    119     EXPLAIN QUERY PLAN SELECT rowid FROM t4 WHERE a=? AND b=?;
   120    120   } {/a=. AND b=./}
          121  +
          122  +#-------------------------------------------------------------------------
          123  +# Test the following case:
          124  +#
          125  +#   ... FROM t1, t2 WHERE (
          126  +#     t2.rowid = +t1.rowid OR (t2.f2 = t1.f1 AND t1.f1!=-1)
          127  +#   )
          128  +#
          129  +# where there is an index on t2(f2). The planner should use "t1" as the
          130  +# outer loop. The inner loop, on "t2", is an OR optimization. One pass
          131  +# for:
          132  +#
          133  +#     t2.rowid = $1
          134  +#
          135  +# and another for:
          136  +#
          137  +#     t2.f2=$1 AND $1!=-1
          138  +#
          139  +# the test is to ensure that on the second pass, the ($1!=-1) condition
          140  +# is tested before any seek operations are performed - i.e. outside of
          141  +# the loop through the f2=$1 range of the t2(f2) index.
          142  +#
          143  +reset_db
          144  +do_execsql_test 5.0 {
          145  +  CREATE TABLE t1(f1);
          146  +  CREATE TABLE t2(f2);
          147  +  CREATE INDEX t2f ON t2(f2);
          148  +
          149  +  INSERT INTO t1 VALUES(-1);
          150  +  INSERT INTO t1 VALUES(-1);
          151  +  INSERT INTO t1 VALUES(-1);
          152  +  INSERT INTO t1 VALUES(-1);
          153  +
          154  +  WITH w(i) AS (
          155  +    SELECT 1 UNION ALL SELECT i+1 FROM w WHERE i<1000
          156  +  )
          157  +  INSERT INTO t2 SELECT -1 FROM w;
          158  +}
          159  +
          160  +do_execsql_test 5.1 {
          161  +  SELECT count(*) FROM t1, t2 WHERE t2.rowid = +t1.rowid
          162  +} {4}
          163  +do_test 5.2 { expr [db status vmstep]<200 } 1
          164  +
          165  +do_execsql_test 5.3 {
          166  +  SELECT count(*) FROM t1, t2 WHERE (
          167  +    t2.rowid = +t1.rowid OR t2.f2 = t1.f1
          168  +  )
          169  +} {4000}
          170  +do_test 5.4 { expr [db status vmstep]>1000 } 1
          171  +
          172  +do_execsql_test 5.5 {
          173  +  SELECT count(*) FROM t1, t2 WHERE (
          174  +    t2.rowid = +t1.rowid OR (t2.f2 = t1.f1 AND t1.f1!=-1)
          175  +  )
          176  +} {4}
          177  +do_test 5.6 { expr [db status vmstep]<200 } 1
   121    178   
   122    179   finish_test

Changes to test/with1.test.

   985    985     WITH xyz(x) AS (
   986    986       SELECT printf('%d', 5) * NULL
   987    987       UNION SELECT round(1<1+x) 
   988    988       FROM xyz ORDER BY 1
   989    989     )
   990    990     SELECT 1 FROM xyz;
   991    991   } 1
          992  +
          993  +# EXPLAIN QUERY PLAN on a self-join of a CTE
          994  +#
          995  +do_execsql_test 19.1 {
          996  +  DROP TABLE IF EXISTS t1;
          997  +  CREATE TABLE t1(x);
          998  +  EXPLAIN QUERY PLAN
          999  +  WITH
         1000  +    x1(a) AS (values(100))
         1001  +  INSERT INTO t1(x)
         1002  +    SELECT * FROM (WITH x2(y) AS (SELECT * FROM x1) SELECT y+a FROM x1, x2);
         1003  +  SELECT * FROM t1;
         1004  +} {0 0 0 {SCAN SUBQUERY 1} 0 1 1 {SCAN SUBQUERY 1}}
         1005  +
   992   1006   
   993   1007   
   994   1008   finish_test

Changes to tool/mkautoconfamal.sh.

    47     47   rm -rf $TMPSPACE
    48     48   cp -R $TOP/autoconf       $TMPSPACE
    49     49   cp sqlite3.c              $TMPSPACE
    50     50   cp sqlite3.h              $TMPSPACE
    51     51   cp sqlite3ext.h           $TMPSPACE
    52     52   cp $TOP/sqlite3.1         $TMPSPACE
    53     53   cp $TOP/sqlite3.pc.in     $TMPSPACE
    54         -cp $TOP/src/msvc.h        $TMPSPACE
    55     54   cp $TOP/src/shell.c       $TMPSPACE
    56     55   cp $TOP/src/sqlite3.rc    $TMPSPACE
    57     56   cp $TOP/tool/Replace.cs   $TMPSPACE
    58     57   
    59     58   cat $TMPSPACE/configure.ac |
    60     59   sed "s/--SQLITE-VERSION--/$VERSION/" > $TMPSPACE/tmp
    61     60   mv $TMPSPACE/tmp $TMPSPACE/configure.ac

Added tool/mkctimec.tcl.

            1  +#!/usr/bin/tclsh
            2  +#
            3  +# To build the
            4  +#
            5  +#   const char **azCompileOpt[]
            6  +#
            7  +# declaration used in src/ctime.c, run this script.
            8  +#
            9  +
           10  +# All Boolean compile time options.
           11  +#
           12  +set boolean_options {
           13  +  SQLITE_32BIT_ROWID
           14  +  SQLITE_4_BYTE_ALIGNED_MALLOC
           15  +  SQLITE_64BIT_STATS
           16  +  SQLITE_ALLOW_COVERING_INDEX_SCAN
           17  +  SQLITE_ALLOW_URI_AUTHORITY
           18  +  SQLITE_BUG_COMPATIBLE_20160819
           19  +  SQLITE_CASE_SENSITIVE_LIKE
           20  +  SQLITE_CHECK_PAGES
           21  +  SQLITE_COVERAGE_TEST
           22  +  SQLITE_DEBUG
           23  +  SQLITE_DEFAULT_AUTOMATIC_INDEX
           24  +  SQLITE_DEFAULT_AUTOVACUUM
           25  +  SQLITE_DEFAULT_CKPTFULLFSYNC
           26  +  SQLITE_DEFAULT_FOREIGN_KEYS
           27  +  SQLITE_DEFAULT_LOCKING_MODE
           28  +  SQLITE_DEFAULT_MEMSTATUS
           29  +  SQLITE_DEFAULT_RECURSIVE_TRIGGERS
           30  +  SQLITE_DEFAULT_SYNCHRONOUS
           31  +  SQLITE_DEFAULT_WAL_SYNCHRONOUS
           32  +  SQLITE_DIRECT_OVERFLOW_READ
           33  +  SQLITE_DISABLE_DIRSYNC
           34  +  SQLITE_DISABLE_FTS3_UNICODE
           35  +  SQLITE_DISABLE_FTS4_DEFERRED
           36  +  SQLITE_DISABLE_INTRINSIC
           37  +  SQLITE_DISABLE_LFS
           38  +  SQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS
           39  +  SQLITE_DISABLE_SKIPAHEAD_DISTINCT
           40  +  SQLITE_ENABLE_8_3_NAMES
           41  +  SQLITE_ENABLE_API_ARMOR
           42  +  SQLITE_ENABLE_ATOMIC_WRITE
           43  +  SQLITE_ENABLE_CEROD
           44  +  SQLITE_ENABLE_COLUMN_METADATA
           45  +  SQLITE_ENABLE_COLUMN_USED_MASK
           46  +  SQLITE_ENABLE_COSTMULT
           47  +  SQLITE_ENABLE_CURSOR_HINTS
           48  +  SQLITE_ENABLE_DBSTAT_VTAB
           49  +  SQLITE_ENABLE_EXPENSIVE_ASSERT
           50  +  SQLITE_ENABLE_FTS1
           51  +  SQLITE_ENABLE_FTS2
           52  +  SQLITE_ENABLE_FTS3
           53  +  SQLITE_ENABLE_FTS3_PARENTHESIS
           54  +  SQLITE_ENABLE_FTS3_TOKENIZER
           55  +  SQLITE_ENABLE_FTS4
           56  +  SQLITE_ENABLE_FTS5
           57  +  SQLITE_ENABLE_HIDDEN_COLUMNS
           58  +  SQLITE_ENABLE_ICU
           59  +  SQLITE_ENABLE_IOTRACE
           60  +  SQLITE_ENABLE_JSON1
           61  +  SQLITE_ENABLE_LOAD_EXTENSION
           62  +  SQLITE_ENABLE_LOCKING_STYLE
           63  +  SQLITE_ENABLE_MEMORY_MANAGEMENT
           64  +  SQLITE_ENABLE_MEMSYS3
           65  +  SQLITE_ENABLE_MEMSYS5
           66  +  SQLITE_ENABLE_MULTIPLEX
           67  +  SQLITE_ENABLE_NULL_TRIM
           68  +  SQLITE_ENABLE_OVERSIZE_CELL_CHECK
           69  +  SQLITE_ENABLE_PREUPDATE_HOOK
           70  +  SQLITE_ENABLE_RBU
           71  +  SQLITE_ENABLE_RTREE
           72  +  SQLITE_ENABLE_SELECTTRACE
           73  +  SQLITE_ENABLE_SESSION
           74  +  SQLITE_ENABLE_SNAPSHOT
           75  +  SQLITE_ENABLE_SQLLOG
           76  +  SQLITE_ENABLE_STMT_SCANSTATUS
           77  +  SQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
           78  +  SQLITE_ENABLE_UNLOCK_NOTIFY
           79  +  SQLITE_ENABLE_UPDATE_DELETE_LIMIT
           80  +  SQLITE_ENABLE_URI_00_ERROR
           81  +  SQLITE_ENABLE_VFSTRACE
           82  +  SQLITE_ENABLE_WHERETRACE
           83  +  SQLITE_ENABLE_ZIPVFS
           84  +  SQLITE_EXPLAIN_ESTIMATED_ROWS
           85  +  SQLITE_EXTRA_IFNULLROW
           86  +  SQLITE_FTS5_ENABLE_TEST_MI
           87  +  SQLITE_FTS5_NO_WITHOUT_ROWID
           88  +  SQLITE_HAS_CODEC
           89  +  SQLITE_HOMEGROWN_RECURSIVE_MUTEX
           90  +  SQLITE_IGNORE_AFP_LOCK_ERRORS
           91  +  SQLITE_IGNORE_FLOCK_LOCK_ERRORS
           92  +  SQLITE_INLINE_MEMCPY
           93  +  SQLITE_INT64_TYPE
           94  +  SQLITE_LIKE_DOESNT_MATCH_BLOBS
           95  +  SQLITE_LOCK_TRACE
           96  +  SQLITE_LOG_CACHE_SPILL
           97  +  SQLITE_MEMDEBUG
           98  +  SQLITE_MIXED_ENDIAN_64BIT_FLOAT
           99  +  SQLITE_MMAP_READWRITE
          100  +  SQLITE_MUTEX_NOOP
          101  +  SQLITE_MUTEX_NREF
          102  +  SQLITE_MUTEX_OMIT
          103  +  SQLITE_MUTEX_PTHREADS
          104  +  SQLITE_MUTEX_W32
          105  +  SQLITE_NEED_ERR_NAME
          106  +  SQLITE_NOINLINE
          107  +  SQLITE_NO_SYNC
          108  +  SQLITE_OMIT_ALTERTABLE
          109  +  SQLITE_OMIT_ANALYZE
          110  +  SQLITE_OMIT_ATTACH
          111  +  SQLITE_OMIT_AUTHORIZATION
          112  +  SQLITE_OMIT_AUTOINCREMENT
          113  +  SQLITE_OMIT_AUTOINIT
          114  +  SQLITE_OMIT_AUTOMATIC_INDEX
          115  +  SQLITE_OMIT_AUTORESET
          116  +  SQLITE_OMIT_AUTOVACUUM
          117  +  SQLITE_OMIT_BETWEEN_OPTIMIZATION
          118  +  SQLITE_OMIT_BLOB_LITERAL
          119  +  SQLITE_OMIT_BTREECOUNT
          120  +  SQLITE_OMIT_CAST
          121  +  SQLITE_OMIT_CHECK
          122  +  SQLITE_OMIT_COMPLETE
          123  +  SQLITE_OMIT_COMPOUND_SELECT
          124  +  SQLITE_OMIT_CONFLICT_CLAUSE
          125  +  SQLITE_OMIT_CTE
          126  +  SQLITE_OMIT_DATETIME_FUNCS
          127  +  SQLITE_OMIT_DECLTYPE
          128  +  SQLITE_OMIT_DEPRECATED
          129  +  SQLITE_OMIT_DISKIO
          130  +  SQLITE_OMIT_EXPLAIN
          131  +  SQLITE_OMIT_FLAG_PRAGMAS
          132  +  SQLITE_OMIT_FLOATING_POINT
          133  +  SQLITE_OMIT_FOREIGN_KEY
          134  +  SQLITE_OMIT_GET_TABLE
          135  +  SQLITE_OMIT_HEX_INTEGER
          136  +  SQLITE_OMIT_INCRBLOB
          137  +  SQLITE_OMIT_INTEGRITY_CHECK
          138  +  SQLITE_OMIT_LIKE_OPTIMIZATION
          139  +  SQLITE_OMIT_LOAD_EXTENSION
          140  +  SQLITE_OMIT_LOCALTIME
          141  +  SQLITE_OMIT_LOOKASIDE
          142  +  SQLITE_OMIT_MEMORYDB
          143  +  SQLITE_OMIT_OR_OPTIMIZATION
          144  +  SQLITE_OMIT_PAGER_PRAGMAS
          145  +  SQLITE_OMIT_PARSER_TRACE
          146  +  SQLITE_OMIT_POPEN
          147  +  SQLITE_OMIT_PRAGMA
          148  +  SQLITE_OMIT_PROGRESS_CALLBACK
          149  +  SQLITE_OMIT_QUICKBALANCE
          150  +  SQLITE_OMIT_REINDEX
          151  +  SQLITE_OMIT_SCHEMA_PRAGMAS
          152  +  SQLITE_OMIT_SCHEMA_VERSION_PRAGMAS
          153  +  SQLITE_OMIT_SHARED_CACHE
          154  +  SQLITE_OMIT_SHUTDOWN_DIRECTORIES
          155  +  SQLITE_OMIT_SUBQUERY
          156  +  SQLITE_OMIT_TCL_VARIABLE
          157  +  SQLITE_OMIT_TEMPDB
          158  +  SQLITE_OMIT_TEST_CONTROL
          159  +  SQLITE_OMIT_TRACE
          160  +  SQLITE_OMIT_TRIGGER
          161  +  SQLITE_OMIT_TRUNCATE_OPTIMIZATION
          162  +  SQLITE_OMIT_UTF16
          163  +  SQLITE_OMIT_VACUUM
          164  +  SQLITE_OMIT_VIEW
          165  +  SQLITE_OMIT_VIRTUALTABLE
          166  +  SQLITE_OMIT_WAL
          167  +  SQLITE_OMIT_WSD
          168  +  SQLITE_OMIT_XFER_OPT
          169  +  SQLITE_PCACHE_SEPARATE_HEADER
          170  +  SQLITE_PERFORMANCE_TRACE
          171  +  SQLITE_POWERSAFE_OVERWRITE
          172  +  SQLITE_PREFER_PROXY_LOCKING
          173  +  SQLITE_PROXY_DEBUG
          174  +  SQLITE_REVERSE_UNORDERED_SELECTS
          175  +  SQLITE_RTREE_INT_ONLY
          176  +  SQLITE_SECURE_DELETE
          177  +  SQLITE_SMALL_STACK
          178  +  SQLITE_SOUNDEX
          179  +  SQLITE_SUBSTR_COMPATIBILITY
          180  +  SQLITE_SYSTEM_MALLOC
          181  +  SQLITE_TCL
          182  +  SQLITE_TEST
          183  +  SQLITE_UNLINK_AFTER_CLOSE
          184  +  SQLITE_UNTESTABLE
          185  +  SQLITE_USE_ALLOCA
          186  +  SQLITE_USE_FCNTL_TRACE
          187  +  SQLITE_USER_AUTHENTICATION
          188  +  SQLITE_USE_URI
          189  +  SQLITE_VDBE_COVERAGE
          190  +  SQLITE_WIN32_MALLOC
          191  +  SQLITE_ZERO_MALLOC
          192  +}
          193  +
          194  +# All compile time options for which the assigned value is other than boolean.
          195  +#
          196  +set value_options {
          197  +  SQLITE_BITMASK_TYPE
          198  +  SQLITE_DEFAULT_CACHE_SIZE
          199  +  SQLITE_DEFAULT_FILE_FORMAT
          200  +  SQLITE_DEFAULT_FILE_PERMISSIONS
          201  +  SQLITE_DEFAULT_JOURNAL_SIZE_LIMIT
          202  +  SQLITE_DEFAULT_LOCKING_MODE
          203  +  SQLITE_DEFAULT_LOOKASIDE
          204  +  SQLITE_DEFAULT_MMAP_SIZE
          205  +  SQLITE_DEFAULT_PAGE_SIZE
          206  +  SQLITE_DEFAULT_PCACHE_INITSZ
          207  +  SQLITE_DEFAULT_PROXYDIR_PERMISSIONS
          208  +  SQLITE_DEFAULT_ROWEST
          209  +  SQLITE_DEFAULT_SECTOR_SIZE
          210  +  SQLITE_DEFAULT_SYNCHRONOUS
          211  +  SQLITE_DEFAULT_WAL_AUTOCHECKPOINT
          212  +  SQLITE_DEFAULT_WAL_SYNCHRONOUS
          213  +  SQLITE_DEFAULT_WORKER_THREADS
          214  +  SQLITE_ENABLE_8_3_NAMES
          215  +  SQLITE_ENABLE_LOCKING_STYLE
          216  +  SQLITE_EXTRA_INIT
          217  +  SQLITE_EXTRA_SHUTDOWN
          218  +  SQLITE_FTS3_MAX_EXPR_DEPTH
          219  +  SQLITE_INTEGRITY_CHECK_ERROR_MAX
          220  +  SQLITE_MALLOC_SOFT_LIMIT
          221  +  SQLITE_MAX_ATTACHED
          222  +  SQLITE_MAX_COLUMN
          223  +  SQLITE_MAX_COMPOUND_SELECT
          224  +  SQLITE_MAX_DEFAULT_PAGE_SIZE
          225  +  SQLITE_MAX_EXPR_DEPTH
          226  +  SQLITE_MAX_FUNCTION_ARG
          227  +  SQLITE_MAX_LENGTH
          228  +  SQLITE_MAX_LIKE_PATTERN_LENGTH
          229  +  SQLITE_MAX_MEMORY
          230  +  SQLITE_MAX_MMAP_SIZE
          231  +  SQLITE_MAX_MMAP_SIZE_
          232  +  SQLITE_MAX_PAGE_COUNT
          233  +  SQLITE_MAX_PAGE_SIZE
          234  +  SQLITE_MAX_SCHEMA_RETRY
          235  +  SQLITE_MAX_SQL_LENGTH
          236  +  SQLITE_MAX_TRIGGER_DEPTH
          237  +  SQLITE_MAX_VARIABLE_NUMBER
          238  +  SQLITE_MAX_VDBE_OP
          239  +  SQLITE_MAX_WORKER_THREADS
          240  +  SQLITE_SORTER_PMASZ
          241  +  SQLITE_STAT4_SAMPLES
          242  +  SQLITE_STMTJRNL_SPILL
          243  +  SQLITE_TEMP_STORE
          244  +}
          245  +
          246  +# Options that require custom code.
          247  +#
          248  +set options(ENABLE_STAT3) {
          249  +#if defined(SQLITE_ENABLE_STAT4)
          250  +  "ENABLE_STAT4",
          251  +#elif defined(SQLITE_ENABLE_STAT3)
          252  +  "ENABLE_STAT3",
          253  +#endif
          254  +}
          255  +set options(COMPILER) {
          256  +#if defined(__clang__) && defined(__clang_major__)
          257  +  "COMPILER=clang-" CTIMEOPT_VAL(__clang_major__) "."
          258  +                    CTIMEOPT_VAL(__clang_minor__) "."
          259  +                    CTIMEOPT_VAL(__clang_patchlevel__),
          260  +#elif defined(_MSC_VER)
          261  +  "COMPILER=msvc-" CTIMEOPT_VAL(_MSC_VER),
          262  +#elif defined(__GNUC__) && defined(__VERSION__)
          263  +  "COMPILER=gcc-" __VERSION__,
          264  +#endif
          265  +}
          266  +set options(HAVE_ISNAN) {
          267  +#if HAVE_ISNAN || SQLITE_HAVE_ISNAN
          268  +  "HAVE_ISNAN",
          269  +#endif
          270  +}
          271  +set options(THREADSAFE) {
          272  +#if defined(SQLITE_THREADSAFE)
          273  +  "THREADSAFE=" CTIMEOPT_VAL(SQLITE_THREADSAFE),
          274  +#elif defined(THREADSAFE)
          275  +  "THREADSAFE=" CTIMEOPT_VAL(THREADSAFE),
          276  +#else
          277  +  "THREADSAFE=1"
          278  +#endif
          279  +}
          280  +
          281  +proc trim_name {in} {
          282  +  set ret $in
          283  +  if {[string range $in 0 6]=="SQLITE_"} {
          284  +    set ret [string range $in 7 end]
          285  +  }
          286  +  return $ret
          287  +}
          288  +
          289  +foreach b $boolean_options {
          290  +  set name [trim_name $b]
          291  +  set options($name) [subst {
          292  +#if $b
          293  +  "$name",
          294  +#endif
          295  +}]
          296  +}
          297  +  
          298  +foreach v $value_options {
          299  +  set name [trim_name $v]
          300  +  set options($name) [subst {
          301  +#ifdef $v
          302  +  "$name=" CTIMEOPT_VAL($v),
          303  +#endif
          304  +}]
          305  +}
          306  +
          307  +foreach o [lsort [array names options]] {
          308  +  puts [string trim $options($o)]
          309  +}
          310  +
          311  +

Changes to tool/mkpragmatab.tcl.

   259    259   
   260    260     NAME: foreign_key_list
   261    261     FLAG: NeedSchema Result1 SchemaOpt
   262    262     COLS: id seq table from to on_update on_delete match
   263    263     IF:   !defined(SQLITE_OMIT_FOREIGN_KEY)
   264    264   
   265    265     NAME: foreign_key_check
   266         -  FLAG: NeedSchema
          266  +  FLAG: NeedSchema Result0
   267    267     COLS: table rowid parent fkid
   268    268     IF:   !defined(SQLITE_OMIT_FOREIGN_KEY) && !defined(SQLITE_OMIT_TRIGGER)
   269    269   
   270    270     NAME: parser_trace
   271    271     IF:   defined(SQLITE_DEBUG) && !defined(SQLITE_OMIT_PARSER_TRACE)
   272    272   
   273    273     NAME: case_sensitive_like
   274    274     FLAG: NoColumns
   275    275   
   276    276     NAME: integrity_check
   277         -  FLAG: NeedSchema
          277  +  FLAG: NeedSchema Result0 Result1
   278    278     IF:   !defined(SQLITE_OMIT_INTEGRITY_CHECK)
   279    279   
   280    280     NAME: quick_check
   281    281     TYPE: INTEGRITY_CHECK
   282         -  FLAG: NeedSchema
          282  +  FLAG: NeedSchema Result0 Result1
   283    283     IF:   !defined(SQLITE_OMIT_INTEGRITY_CHECK)
   284    284   
   285    285     NAME: encoding
   286    286     FLAG: Result0 NoColumns1
   287    287     IF:   !defined(SQLITE_OMIT_UTF16)
   288    288   
   289    289     NAME: schema_version

Changes to tool/mksqlite3c.tcl.

   279    279   
   280    280   
   281    281   # Process the source files.  Process files containing commonly
   282    282   # used subroutines first in order to help the compiler find
   283    283   # inlining opportunities.
   284    284   #
   285    285   foreach file {
          286  +   ctime.c
   286    287      sqliteInt.h
   287    288   
   288    289      global.c
   289         -   ctime.c
   290    290      status.c
   291    291      date.c
   292    292      os.c
   293    293   
   294    294      fault.c
   295    295      mem0.c
   296    296      mem1.c

Changes to tool/spaceanal.tcl.

   420    420     # total_pages_percent: Pages consumed as a percentage of the file.
   421    421     # storage: Bytes consumed.
   422    422     # payload_percent: Payload bytes used as a percentage of $storage.
   423    423     # total_unused: Unused bytes on pages.
   424    424     # avg_payload: Average payload per btree entry.
   425    425     # avg_fanout: Average fanout for internal pages.
   426    426     # avg_unused: Average unused bytes per btree entry.
          427  +  # avg_meta: Average metadata overhead per entry.
   427    428     # ovfl_cnt_percent: Percentage of btree entries that use overflow pages.
   428    429     #
   429    430     set total_pages [expr {$leaf_pages+$int_pages+$ovfl_pages}]
   430    431     set total_pages_percent [percent $total_pages $file_pgcnt]
   431    432     set storage [expr {$total_pages*$pageSize}]
   432    433     set payload_percent [percent $payload $storage {of storage consumed}]
   433    434     set total_unused [expr {$ovfl_unused+$int_unused+$leaf_unused}]
   434    435     set avg_payload [divide $payload $nentry]
   435    436     set avg_unused [divide $total_unused $nentry]
          437  +  set total_meta [expr {$storage - $payload - $total_unused}]
          438  +  set total_meta [expr {$total_meta + 4*($ovfl_pages - $ovfl_cnt)}]
          439  +  set meta_percent [percent $total_meta $storage {of metadata}]
          440  +  set avg_meta [divide $total_meta $nentry]
   436    441     if {$int_pages>0} {
   437    442       # TODO: Is this formula correct?
   438    443       set nTab [mem eval "
   439    444         SELECT count(*) FROM (
   440    445             SELECT DISTINCT tblname FROM space_used WHERE $where AND is_index=0
   441    446         )
   442    447       "]
................................................................................
   456    461     if {$compressed_size!=$storage} {
   457    462       set compressed_size [expr {$compressed_size+$compressOverhead*$total_pages}]
   458    463       set pct [expr {$compressed_size*100.0/$storage}]
   459    464       set pct [format {%5.1f%%} $pct]
   460    465       statline {Bytes used after compression} $compressed_size $pct
   461    466     }
   462    467     statline {Bytes of payload} $payload $payload_percent
          468  +  statline {Bytes of metadata} $total_meta $meta_percent
   463    469     if {$cnt==1} {statline {B-tree depth} $depth}
   464    470     statline {Average payload per entry} $avg_payload
   465    471     statline {Average unused bytes per entry} $avg_unused
          472  +  statline {Average metadata per entry} $avg_meta
   466    473     if {[info exists avg_fanout]} {
   467    474       statline {Average fanout} $avg_fanout
   468    475     }
   469    476     if {$showFrag && $total_pages>1} {
   470    477       set fragmentation [percent $gap_cnt [expr {$total_pages-1}]]
   471    478       statline {Non-sequential pages} $gap_cnt $fragmentation
   472    479     }
................................................................................
   753    760   Bytes of payload
   754    761   
   755    762       The amount of payload stored under this category.  Payload is the data
   756    763       part of table entries and the key part of index entries.  The percentage
   757    764       at the right is the bytes of payload divided by the bytes of storage 
   758    765       consumed.
   759    766   
          767  +Bytes of metadata
          768  +
          769  +    The amount of formatting and structural information stored in the
          770  +    table or index.  Metadata includes the btree page header, the cell pointer
          771  +    array, the size field for each cell, the left child pointer or non-leaf
          772  +    cells, the overflow pointers for overflow cells, and the rowid value for
          773  +    rowid table cells.  In other words, metadata is everything that is neither
          774  +    unused space nor content.  The record header in the payload is counted as
          775  +    content, not metadata.
          776  +
   760    777   Average payload per entry
   761    778   
   762    779       The average amount of payload on each entry.  This is just the bytes of
   763    780       payload divided by the number of entries.
   764    781   
   765    782   Average unused bytes per entry
   766    783   

Changes to tool/sqldiff.c.

  1663   1663     if( g.fDebug & DEBUG_DIFF_SQL ){ 
  1664   1664       printf("SQL for %s:\n%s\n", zId, sql.z);
  1665   1665       goto end_changeset_one_table;
  1666   1666     }
  1667   1667   
  1668   1668     putc('T', out);
  1669   1669     putsVarint(out, (sqlite3_uint64)nCol);
  1670         -  for(i=0; i<nCol; i++) putc(aiFlg[i]!=0, out);
         1670  +  for(i=0; i<nCol; i++) putc(aiFlg[i], out);
  1671   1671     fwrite(zTab, 1, strlen(zTab), out);
  1672   1672     putc(0, out);
  1673   1673   
  1674   1674     pStmt = db_prepare("%s", sql.z);
  1675   1675     while( SQLITE_ROW==sqlite3_step(pStmt) ){
  1676   1676       int iType = sqlite3_column_int(pStmt,0);
  1677   1677       putc(iType, out);