SQLite

Changes On Branch oo1-no-cache-Stmt.columnCount
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch oo1-no-cache-Stmt.columnCount Excluding Merge-Ins

This is equivalent to a diff from 4150e416 to e841b142

2023-05-17
00:47
Resolve the JS-side corner case reported in forum post 7774b773937cbe0a by not caching oo1.Stmt.columnCount. (check-in: adee1440 user: stephan tags: branch-3.42)
2023-05-16
14:18
Resolve the JS-side corner case reported in forum post 7774b773937cbe0a by not caching oo1.Stmt.columnCount. (check-in: 37d44e8c user: stephan tags: trunk)
2023-05-11
17:25
Fix the rbu_exclusive_checkpoint=1 option so that it works with a zipvfs database in pass-through mode. (check-in: caca67d3 user: dan tags: trunk)
2023-05-10
21:08
Doc typo fix. No code changes. (Closed-Leaf check-in: e841b142 user: stephan tags: oo1-no-cache-Stmt.columnCount)
21:06
Resolve the JS-side corner case reported in forum post 7774b773937cbe0a by not caching oo1.Stmt.columnCount. (check-in: c3d25c3a user: stephan tags: oo1-no-cache-Stmt.columnCount)
16:04
Enhance the split-sqlite3c.tcl script so that it uses single-character extensions on subordinate source files, even when the number of subordinates exceeds 9. This is not yet needed, but might be soon. (check-in: 4150e416 user: drh tags: trunk)
13:56
Fix harmless compiler warnings from the CBMC compiler. (check-in: 5bc17cbc user: drh tags: trunk)

Changes to ext/wasm/api/sqlite3-api-oo1.js.

326
327
328
329
330
331
332
333
334





335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
     functions, its read-only `pointer` property holds its `sqlite3_stmt*`
     pointer value.

     Other non-function properties include:

     - `db`: the DB object which created the statement.

     - `columnCount`: the number of result columns in the query, or 0 for
     queries which cannot return results.






     - `parameterCount`: the number of bindable paramters in the query.
  */
  const Stmt = function(){
    if(BindTypes!==arguments[2]){
      toss3(capi.SQLITE_MISUSE, "Do not call the Stmt constructor directly. Use DB.prepare().");
    }
    this.db = arguments[0];
    __ptrMap.set(this, arguments[1]);
    this.columnCount = capi.sqlite3_column_count(this.pointer);
    this.parameterCount = capi.sqlite3_bind_parameter_count(this.pointer);
  };

  /** Throws if the given DB has been closed, else it is returned. */
  const affirmDbOpen = function(db){
    if(!db.pointer) toss3("DB has been closed.");
    return db;







|
|
>
>
>
>
>

|







<







326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348

349
350
351
352
353
354
355
     functions, its read-only `pointer` property holds its `sqlite3_stmt*`
     pointer value.

     Other non-function properties include:

     - `db`: the DB object which created the statement.

     - `columnCount`: the number of result columns in the query, or 0
     for queries which cannot return results. This property is a proxy
     for sqlite3_column_count() and its use in loops should be avoided
     because of the call overhead associated with that. The
     `columnCount` is not cached when the Stmt is created because a
     schema change made via a separate db connection between this
     statement's preparation and when it is stepped may invalidate it.

     - `parameterCount`: the number of bindable parameters in the query.
  */
  const Stmt = function(){
    if(BindTypes!==arguments[2]){
      toss3(capi.SQLITE_MISUSE, "Do not call the Stmt constructor directly. Use DB.prepare().");
    }
    this.db = arguments[0];
    __ptrMap.set(this, arguments[1]);

    this.parameterCount = capi.sqlite3_bind_parameter_count(this.pointer);
  };

  /** Throws if the given DB has been closed, else it is returned. */
  const affirmDbOpen = function(db){
    if(!db.pointer) toss3("DB has been closed.");
    return db;
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
       result set are stored in this array before the callback (if
       any) is triggered (regardless of whether the query produces any
       result rows). If no statement has result columns, this value is
       unchanged. Achtung: an SQL result may have multiple columns
       with identical names.

       - `callback` = a function which gets called for each row of the
       result set, but only if that statement has any result
       _rows_. The callback's "this" is the options object, noting
       that this function synthesizes one if the caller does not pass
       one to exec(). The second argument passed to the callback is
       always the current Stmt object, as it's needed if the caller
       wants to fetch the column names or some such (noting that they
       could also be fetched via `this.columnNames`, if the client
       provides the `columnNames` option). If the callback returns a
       literal `false` (as opposed to any other falsy value, e.g.  an
       implicit `undefined` return), any ongoing statement-`step()`
       iteration stops without an error. The return value of the
       callback is otherwise ignored.

       ACHTUNG: The callback MUST NOT modify the Stmt object. Calling
       any of the Stmt.get() variants, Stmt.getColumnName(), or
       similar, is legal, but calling step() or finalize() is
       not. Member methods which are illegal in this context will
       trigger an exception, but clients must also refrain from using
       any lower-level (C-style) APIs which might modify the
       statement.

       The first argument passed to the callback defaults to an array of
       values from the current result row but may be changed with ...

       - `rowMode` = specifies the type of he callback's first argument.
       It may be any of...

       A) A string describing what type of argument should be passed
       as the first argument to the callback:

         A.1) `'array'` (the default) causes the results of
         `stmt.get([])` to be passed to the `callback` and/or appended
         to `resultRows`

         A.2) `'object'` causes the results of
         `stmt.get(Object.create(null))` to be passed to the
         `callback` and/or appended to `resultRows`.  Achtung: an SQL
         result may have multiple columns with identical names. In
         that case, the right-most column will be the one set in this
         object!

         A.3) `'stmt'` causes the current Stmt to be passed to the
         callback, but this mode will trigger an exception if
         `resultRows` is an array because appending the statement to
         the array would be downright unhelpful.

       B) An integer, indicating a zero-based column in the result
       row. Only that one single value will be passed on.

       C) A string with a minimum length of 2 and leading character of
       '$' will fetch the row as an object, extract that one field,
       and pass that field's value to the callback. Note that these







|
|
|
|
|
|
|
|
|
|
|
|




















|










|
|







701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
       result set are stored in this array before the callback (if
       any) is triggered (regardless of whether the query produces any
       result rows). If no statement has result columns, this value is
       unchanged. Achtung: an SQL result may have multiple columns
       with identical names.

       - `callback` = a function which gets called for each row of the
       result set, but only if that statement has any result rows. The
       callback's "this" is the options object, noting that this
       function synthesizes one if the caller does not pass one to
       exec(). The second argument passed to the callback is always
       the current Stmt object, as it's needed if the caller wants to
       fetch the column names or some such (noting that they could
       also be fetched via `this.columnNames`, if the client provides
       the `columnNames` option). If the callback returns a literal
       `false` (as opposed to any other falsy value, e.g. an implicit
       `undefined` return), any ongoing statement-`step()` iteration
       stops without an error. The return value of the callback is
       otherwise ignored.

       ACHTUNG: The callback MUST NOT modify the Stmt object. Calling
       any of the Stmt.get() variants, Stmt.getColumnName(), or
       similar, is legal, but calling step() or finalize() is
       not. Member methods which are illegal in this context will
       trigger an exception, but clients must also refrain from using
       any lower-level (C-style) APIs which might modify the
       statement.

       The first argument passed to the callback defaults to an array of
       values from the current result row but may be changed with ...

       - `rowMode` = specifies the type of he callback's first argument.
       It may be any of...

       A) A string describing what type of argument should be passed
       as the first argument to the callback:

         A.1) `'array'` (the default) causes the results of
         `stmt.get([])` to be passed to the `callback` and/or appended
         to `resultRows`.

         A.2) `'object'` causes the results of
         `stmt.get(Object.create(null))` to be passed to the
         `callback` and/or appended to `resultRows`.  Achtung: an SQL
         result may have multiple columns with identical names. In
         that case, the right-most column will be the one set in this
         object!

         A.3) `'stmt'` causes the current Stmt to be passed to the
         callback, but this mode will trigger an exception if
         `resultRows` is an array because appending the transient
         statement to the array would be downright unhelpful.

       B) An integer, indicating a zero-based column in the result
       row. Only that one single value will be passed on.

       C) A string with a minimum length of 2 and leading character of
       '$' will fetch the row as an object, extract that one field,
       and pass that field's value to the callback. Note that these
853
854
855
856
857
858
859
860
861









862
863
864
865
866
867

868
869
870
871
872
873
874
875
876




877
878
879
880
881
882
883
          if(saveSql) saveSql.push(capi.sqlite3_sql(pStmt).trim());
          stmt = new Stmt(this, pStmt, BindTypes);
          if(bind && stmt.parameterCount){
            stmt.bind(bind);
            bind = null;
          }
          if(evalFirstResult && stmt.columnCount){
            /* Only forward SELECT results for the FIRST query
               in the SQL which potentially has them. */









            evalFirstResult = false;
            if(Array.isArray(opt.columnNames)){
              stmt.getColumnNames(opt.columnNames);
            }
            if(arg.cbArg || resultRows){
              for(; stmt.step(); stmt._isLocked = false){

                stmt._isLocked = true;
                const row = arg.cbArg(stmt);
                if(resultRows) resultRows.push(row);
                if(callback && false === callback.call(opt, row, stmt)){
                  break;
                }
              }
              stmt._isLocked = false;
            }




          }else{
            stmt.step();
          }
          stmt.finalize();
          stmt = null;
        }
      }/*catch(e){







|

>
>
>
>
>
>
>
>
>

<
<
<


>









>
>
>
>







857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875



876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
          if(saveSql) saveSql.push(capi.sqlite3_sql(pStmt).trim());
          stmt = new Stmt(this, pStmt, BindTypes);
          if(bind && stmt.parameterCount){
            stmt.bind(bind);
            bind = null;
          }
          if(evalFirstResult && stmt.columnCount){
            /* Only forward SELECT-style results for the FIRST query
               in the SQL which potentially has them. */
            let gotColNames = Array.isArray(
              opt.columnNames
              /* As reported in
                 https://sqlite.org/forum/forumpost/7774b773937cbe0a
                 we need to delay fetching of the column names until
                 after the first step() (if we step() at all) because
                 a schema change between the prepare() and step(), via
                 another connection, may invalidate the column count
                 and names. */) ? 0 : 1;
            evalFirstResult = false;



            if(arg.cbArg || resultRows){
              for(; stmt.step(); stmt._isLocked = false){
                if(0===gotColNames++) stmt.getColumnNames(opt.columnNames);
                stmt._isLocked = true;
                const row = arg.cbArg(stmt);
                if(resultRows) resultRows.push(row);
                if(callback && false === callback.call(opt, row, stmt)){
                  break;
                }
              }
              stmt._isLocked = false;
            }
            if(0===gotColNames){
              /* opt.columnNames was provided but we visited no result rows */
              stmt.getColumnNames(opt.columnNames);
            }
          }else{
            stmt.step();
          }
          stmt.finalize();
          stmt = null;
        }
      }/*catch(e){
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
    finalize: function(){
      if(this.pointer){
        affirmUnlocked(this,'finalize()');
        delete __stmtMap.get(this.db)[this.pointer];
        capi.sqlite3_finalize(this.pointer);
        __ptrMap.delete(this);
        delete this._mayGet;
        delete this.columnCount;
        delete this.parameterCount;
        delete this.db;
        delete this._isLocked;
      }
    },
    /** Clears all bound values. Returns this object.
        Throws if this statement has been finalized. */







<







1427
1428
1429
1430
1431
1432
1433

1434
1435
1436
1437
1438
1439
1440
    finalize: function(){
      if(this.pointer){
        affirmUnlocked(this,'finalize()');
        delete __stmtMap.get(this.db)[this.pointer];
        capi.sqlite3_finalize(this.pointer);
        __ptrMap.delete(this);
        delete this._mayGet;

        delete this.parameterCount;
        delete this.db;
        delete this._isLocked;
      }
    },
    /** Clears all bound values. Returns this object.
        Throws if this statement has been finalized. */
1682
1683
1684
1685
1686
1687
1688

1689
1690
1691
1692
1693
1694

1695
1696
1697
1698
1699
1700
1701
1702
    */
    get: function(ndx,asType){
      if(!affirmStmtOpen(this)._mayGet){
        toss3("Stmt.step() has not (recently) returned true.");
      }
      if(Array.isArray(ndx)){
        let i = 0;

        while(i<this.columnCount){
          ndx[i] = this.get(i++);
        }
        return ndx;
      }else if(ndx && 'object'===typeof ndx){
        let i = 0;

        while(i<this.columnCount){
          ndx[capi.sqlite3_column_name(this.pointer,i)] = this.get(i++);
        }
        return ndx;
      }
      affirmColIndex(this, ndx);
      switch(undefined===asType
             ? capi.sqlite3_column_type(this.pointer, ndx)







>
|





>
|







1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
    */
    get: function(ndx,asType){
      if(!affirmStmtOpen(this)._mayGet){
        toss3("Stmt.step() has not (recently) returned true.");
      }
      if(Array.isArray(ndx)){
        let i = 0;
        const n = this.columnCount;
        while(i<n){
          ndx[i] = this.get(i++);
        }
        return ndx;
      }else if(ndx && 'object'===typeof ndx){
        let i = 0;
        const n = this.columnCount;
        while(i<n){
          ndx[capi.sqlite3_column_name(this.pointer,i)] = this.get(i++);
        }
        return ndx;
      }
      affirmColIndex(this, ndx);
      switch(undefined===asType
             ? capi.sqlite3_column_type(this.pointer, ndx)
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801

1802
1803
1804
1805
1806
1807
1808
1809
    */
    getColumnName: function(ndx){
      return capi.sqlite3_column_name(
        affirmColIndex(affirmStmtOpen(this),ndx).pointer, ndx
      );
    },
    /**
       If this statement potentially has result columns, this
       function returns an array of all such names. If passed an
       array, it is used as the target and all names are appended
       to it. Returns the target array. Throws if this statement
       cannot have result columns. This object's columnCount member
       holds the number of columns.
    */
    getColumnNames: function(tgt=[]){
      affirmColIndex(affirmStmtOpen(this),0);

      for(let i = 0; i < this.columnCount; ++i){
        tgt.push(capi.sqlite3_column_name(this.pointer, i));
      }
      return tgt;
    },
    /**
       If this statement has named bindable parameters and the
       given name matches one, its 1-based bind index is







|
|
|
|
|
|



>
|







1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
    */
    getColumnName: function(ndx){
      return capi.sqlite3_column_name(
        affirmColIndex(affirmStmtOpen(this),ndx).pointer, ndx
      );
    },
    /**
       If this statement potentially has result columns, this function
       returns an array of all such names. If passed an array, it is
       used as the target and all names are appended to it. Returns
       the target array. Throws if this statement cannot have result
       columns. This object's columnCount property holds the number of
       columns.
    */
    getColumnNames: function(tgt=[]){
      affirmColIndex(affirmStmtOpen(this),0);
      const n = this.columnCount;
      for(let i = 0; i < n; ++i){
        tgt.push(capi.sqlite3_column_name(this.pointer, i));
      }
      return tgt;
    },
    /**
       If this statement has named bindable parameters and the
       given name matches one, its 1-based bind index is
1822
1823
1824
1825
1826
1827
1828














1829
1830
1831
1832
1833
1834
1835
      enumerable: true,
      get: function(){return __ptrMap.get(this)},
      set: ()=>toss3("The pointer property is read-only.")
    }
    Object.defineProperty(Stmt.prototype, 'pointer', prop);
    Object.defineProperty(DB.prototype, 'pointer', prop);
  }















  /** The OO API's public namespace. */
  sqlite3.oo1 = {
    DB,
    Stmt
  }/*oo1 object*/;








>
>
>
>
>
>
>
>
>
>
>
>
>
>







1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
      enumerable: true,
      get: function(){return __ptrMap.get(this)},
      set: ()=>toss3("The pointer property is read-only.")
    }
    Object.defineProperty(Stmt.prototype, 'pointer', prop);
    Object.defineProperty(DB.prototype, 'pointer', prop);
  }
  /**
     Stmt.columnCount is an interceptor for sqlite3_column_count().

     This requires an unfortunate performance hit compared to caching
     columnCount when the Stmt is created/prepared (as was done in
     SQLite <=3.42.0), but is necessary in order to handle certain
     corner cases, as described in
     https://sqlite.org/forum/forumpost/7774b773937cbe0a.
  */
  Object.defineProperty(Stmt.prototype, 'columnCount', {
    enumerable: false,
    get: function(){return capi.sqlite3_column_count(this.pointer)},
    set: ()=>toss3("The columnCount property is read-only.")
  });

  /** The OO API's public namespace. */
  sqlite3.oo1 = {
    DB,
    Stmt
  }/*oo1 object*/;

Changes to ext/wasm/common/SqliteTestUtil.js.

152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
        if( ! m ) continue;
        args[decodeURIComponent(m[1])] = (m[3] ? decodeURIComponent(m[3]) : true);
      }
      return args;
    }
  };

  
  /**
     This is a module object for use with the emscripten-installed
     sqlite3InitModule() factory function.
  */
  self.sqlite3TestModule = {
    /**
       Array of functions to call after Emscripten has initialized the







<







152
153
154
155
156
157
158

159
160
161
162
163
164
165
        if( ! m ) continue;
        args[decodeURIComponent(m[1])] = (m[3] ? decodeURIComponent(m[3]) : true);
      }
      return args;
    }
  };


  /**
     This is a module object for use with the emscripten-installed
     sqlite3InitModule() factory function.
  */
  self.sqlite3TestModule = {
    /**
       Array of functions to call after Emscripten has initialized the

Changes to ext/wasm/tester1.c-pp.js.

1225
1226
1227
1228
1229
1230
1231


1232
1233
1234
1235
1236
1237
1238
          .assert(1===this.db.openStatementCount())
          .assert(
            capi.sqlite3_stmt_status(
              st, capi.SQLITE_STMTSTATUS_RUN, 0
            ) === 0)
          .assert(!st._mayGet)
          .assert('a' === st.getColumnName(0))


          .assert(1===st.columnCount)
          .assert(0===st.parameterCount)
          .mustThrow(()=>st.bind(1,null))
          .assert(true===st.step())
          .assert(3 === st.get(0))
          .mustThrow(()=>st.get(1))
          .mustThrow(()=>st.get(0,~capi.SQLITE_INTEGER))







>
>







1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
          .assert(1===this.db.openStatementCount())
          .assert(
            capi.sqlite3_stmt_status(
              st, capi.SQLITE_STMTSTATUS_RUN, 0
            ) === 0)
          .assert(!st._mayGet)
          .assert('a' === st.getColumnName(0))
          .mustThrowMatching(()=>st.columnCount=2,
                             /columnCount property is read-only/)
          .assert(1===st.columnCount)
          .assert(0===st.parameterCount)
          .mustThrow(()=>st.bind(1,null))
          .assert(true===st.step())
          .assert(3 === st.get(0))
          .mustThrow(()=>st.get(1))
          .mustThrow(()=>st.get(0,~capi.SQLITE_INTEGER))
1369
1370
1371
1372
1373
1374
1375
1376


1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388








1389
1390
1391
1392
1393
1394
1395
        columnNames: colNames,
        _myState: 3 /* Accessible from the callback */,
        callback: function(row,stmt){
          ++counter;
          T.assert(
            3 === this._myState
            /* Recall that "this" is the options object. */
          ).assert(


            this.columnNames[0]==='a' && this.columnNames[1]==='b'
            /* options.columnNames is filled out before the first
               Stmt.step(). */
          ).assert(
            (row.a%2 && row.a<6) || 'blob'===row.a
          );
        }
      });
      T.assert(2 === colNames.length)
        .assert('a' === colNames[0])
        .assert(4 === counter)
        .assert(4 === list.length);








      list.length = 0;
      db.exec("SELECT a a, b b FROM t",{
        rowMode: 'array',
        callback: function(row,stmt){
          ++counter;
          T.assert(Array.isArray(row))
            .assert((0===row[1]%2 && row[1]<7)








>
>

<
<









>
>
>
>
>
>
>
>







1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381


1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
        columnNames: colNames,
        _myState: 3 /* Accessible from the callback */,
        callback: function(row,stmt){
          ++counter;
          T.assert(
            3 === this._myState
            /* Recall that "this" is the options object. */
          ).assert(
            this.columnNames===colNames
          ).assert(
            this.columnNames[0]==='a' && this.columnNames[1]==='b'


          ).assert(
            (row.a%2 && row.a<6) || 'blob'===row.a
          );
        }
      });
      T.assert(2 === colNames.length)
        .assert('a' === colNames[0])
        .assert(4 === counter)
        .assert(4 === list.length);
      colNames = [];
      db.exec({
        /* Ensure that columnNames is populated for empty result sets. */
        sql: "SELECT a a, b B FROM t WHERE 0",
        columnNames: colNames
      });
      T.assert(2===colNames.length)
        .assert('a'===colNames[0] && 'B'===colNames[1]);
      list.length = 0;
      db.exec("SELECT a a, b b FROM t",{
        rowMode: 'array',
        callback: function(row,stmt){
          ++counter;
          T.assert(Array.isArray(row))
            .assert((0===row[1]%2 && row[1]<7)
1434
1435
1436
1437
1438
1439
1440

1441
1442
1443
1444
1445
1446
1447
           https://github.com/emscripten-core/emscripten/issues/17391 */
        T.mustThrow(()=>db.selectValue("SELECT "+(Number.MAX_SAFE_INTEGER+1))).
          mustThrow(()=>db.selectValue("SELECT "+(Number.MIN_SAFE_INTEGER-1)));
      }

      let st = db.prepare("update t set b=:b where a='blob'");
      try {

        const ndx = st.getParamIndex(':b');
        T.assert(1===ndx);
        st.bindAsBlob(ndx, "ima blob").reset(true);
      } finally {
        st.finalize();
      }








>







1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
           https://github.com/emscripten-core/emscripten/issues/17391 */
        T.mustThrow(()=>db.selectValue("SELECT "+(Number.MAX_SAFE_INTEGER+1))).
          mustThrow(()=>db.selectValue("SELECT "+(Number.MIN_SAFE_INTEGER-1)));
      }

      let st = db.prepare("update t set b=:b where a='blob'");
      try {
        T.assert(0===st.columnCount);
        const ndx = st.getParamIndex(':b');
        T.assert(1===ndx);
        st.bindAsBlob(ndx, "ima blob").reset(true);
      } finally {
        st.finalize();
      }