SQLite

Check-in [d693c2dddb]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add a more complete test for [76c8435a] and add some commentary about (A) the inability to automatically clean up automatically-generated WASM proxy functions for sqlite3_set_auxdata() destructors and (B) how to deal with (A) to avoid leaking WASM proxy functions.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: d693c2dddbd10a2e0b77893b04b11502e30b768f1b06814105f7f35172845fb9
User & Date: stephan 2025-02-03 14:55:56.185
Context
2025-02-03
17:34
Rework [76c8435a] to eliminate automatic JS-to-WASM function conversions of sqlite3_set_auxdata() destructors because it can leads to leaks on every call of a UDF. This feature never worked before [76c8435a] but fixing it was ill-conceived because of the memory leakage it introduces. WASM function pointers can still be used as destructors in this context. (check-in: 3fb993af0c user: stephan tags: trunk)
16:26
Initial work on a fix for the SAHPool VFS's effectively-no-op digest calculation, as reported in ticket #97 of the downstream npm subproject. This requires more testing alongside databases created before this version to ensure that it's backwards-compatible. (check-in: 9234c33f92 user: stephan tags: sahpool-digest)
15:17
Merge the latest trunk changes into the reuse-schema branch. (check-in: 858163f938 user: drh tags: reuse-schema)
15:07
Merge the latest trunk enhancements into the wal2 branch. (check-in: e2d4c1890a user: drh tags: wal2)
14:59
Merge all the latest trunk enhancements and fixes into the begin-concurrent branch. (check-in: f456a72e0c user: drh tags: begin-concurrent)
14:55
Add a more complete test for [76c8435a] and add some commentary about (A) the inability to automatically clean up automatically-generated WASM proxy functions for sqlite3_set_auxdata() destructors and (B) how to deal with (A) to avoid leaking WASM proxy functions. (check-in: d693c2dddb user: stephan tags: trunk)
14:44
Fix the build process on Windows so that it generates identical sqlite3.c, sqlite3.h, and shell.c files on Windows and Unix. This patch also includes a change to JS bindings that got caught up in the branch. (check-in: 91ef45fc29 user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to ext/wasm/api/sqlite3-api-glue.c-pp.js.
224
225
226
227
228
229
230

























231
232
233
234
235
236
237
      new wasm.xWrap.FuncPtrAdapter({
        name: 'sqlite3_rollback_hook',
        signature: 'v(p)',
        contextKey: (argv)=>argv[0/* sqlite3* */]
      }),
      '*'
    ]],

























    ["sqlite3_set_auxdata", undefined, [
      "sqlite3_context*", "int", "*",
      new wasm.xWrap.FuncPtrAdapter({
        name: 'xDestroyAuxData',
        signature: 'v(p)',
        contextKey: (argv, argIndex)=>argv[0/* sqlite3_context* */]
      })







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
      new wasm.xWrap.FuncPtrAdapter({
        name: 'sqlite3_rollback_hook',
        signature: 'v(p)',
        contextKey: (argv)=>argv[0/* sqlite3* */]
      }),
      '*'
    ]],
    /**
       2025-02-03: We do not have a way to automatically clean up
       destructors which are automatically converted from JS functions
       via the final argument to sqlite3_set_auxdata(). Because of
       that, it is strongly recommended that clients use
       wasm.installFunction() to create such callbacks, then pass that
       pointer to sqlite3_set_auxdata(). Relying on automated
       conversions here will lead to leaks of JS/WASM proxy functions
       because sqlite3_set_auxdata() is frequently called in UDFs.

       The sqlite3.oo1.DB class's onclose handlers can be used for this
       purpose. For example:

       const pAuxDtor = wasm.installFunction('v(p)', function(ptr){
         //free ptr
       });
       myDb.onclose = {
         after: ()=>{
           wasm.uninstallFunction(pAuxDtor);
         }
       };

       Then pass pAuxDtor as the final argument to appropriate
       sqlite3_set_auxdata() calls.
    */
    ["sqlite3_set_auxdata", undefined, [
      "sqlite3_context*", "int", "*",
      new wasm.xWrap.FuncPtrAdapter({
        name: 'xDestroyAuxData',
        signature: 'v(p)',
        contextKey: (argv, argIndex)=>argv[0/* sqlite3_context* */]
      })
1043
1044
1045
1046
1047
1048
1049




1050
1051
1052
1053
1054
1055
1056
      'sqlite3_commit_hook',
      'sqlite3_preupdate_hook',
      'sqlite3_progress_handler',
      'sqlite3_rollback_hook',
      'sqlite3_set_authorizer',
      'sqlite3_trace_v2',
      'sqlite3_update_hook'




    ]) {
      const x = wasm.exports[name];
      if( !x ){
        /* assume it was built without this API */
        continue;
      }
      closeArgs.length = x.length/*==argument count*/







>
>
>
>







1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
      'sqlite3_commit_hook',
      'sqlite3_preupdate_hook',
      'sqlite3_progress_handler',
      'sqlite3_rollback_hook',
      'sqlite3_set_authorizer',
      'sqlite3_trace_v2',
      'sqlite3_update_hook'
      /*
        We do not yet have a way to clean up automatically-converted
        sqlite3_set_auxdata() finalizers.
      */
    ]) {
      const x = wasm.exports[name];
      if( !x ){
        /* assume it was built without this API */
        continue;
      }
      closeArgs.length = x.length/*==argument count*/
Changes to ext/wasm/tester1.c-pp.js.
3432
3433
3434
3435
3436
3437
3438



































































3439
3440
3441
3442
3443
3444
3445
          tryOne(false, "Emscripten filesystem");
          tryOne(poolConfig.name);
          tryOne('opfs');
        }finally{
          await poolUtil.removeVfs();
        }
      }



































































    })
  ;/*end of Bug Reports group*/;

  ////////////////////////////////////////////////////////////////////////
  log("Loading and initializing sqlite3 WASM module...");
  if(0){
    globalThis.sqlite3ApiConfig = {







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491
3492
3493
3494
3495
3496
3497
3498
3499
3500
3501
3502
3503
3504
3505
3506
3507
3508
3509
3510
3511
3512
          tryOne(false, "Emscripten filesystem");
          tryOne(poolConfig.name);
          tryOne('opfs');
        }finally{
          await poolUtil.removeVfs();
        }
      }
    })
    .t({
      /* https://github.com/sqlite/sqlite-wasm/issues/92 */
      name: 'sqlite3_set_auxdata() binding signature',
      test: function(sqlite3){
        const db = new sqlite3.oo1.DB();
        const stack = wasm.pstack.pointer;
        const pAux = wasm.pstack.alloc(4);
        let pAuxDestructed = 0;
        const args = [];
        const pAuxDtor = wasm.installFunction('v(p)', function(ptr){
          //log("freeing auxdata");
          ++pAuxDestructed;
        });
        let pAuxDtorDestructed = false;
        db.onclose = {
          after: ()=>{
            pAuxDtorDestructed = true;
            wasm.uninstallFunction(pAuxDtor);
          }
        };
        try{
          db.createFunction("auxtest",{
            xFunc: function(pCx, x, y){
              args.push(x);
              T.assert(wasm.isPtr(pCx));
              const localAux = capi.sqlite3_get_auxdata(pCx, 0);
              if( !localAux ){
                //log("setting auxdata");
                /**
                   We do not currently an automated way to clean up
                   auxdata finalizer functions (the 4th argument to
                   sqlite3_set_auxdata()) which get automatically
                   converted from JS to WASM. Because of that, relying
                   on automated conversions for those is not
                   recommended. Instead, follow the pattern show in
                   this function: use wasm.installFunction() to create
                   the function, then pass the resulting function
                   pointer this function, and cleanup (at some point)
                   using wasm.uninstallFunction().
                */
                capi.sqlite3_set_auxdata(pCx, 0, pAux, pAuxDtor);
              }else{
                /* This is never actually hit in this example and it's
                   not entirely clear how to cause it to. The point of
                   this test, however, is to demonstrate that the
                   finalizer impl gets triggered, so we're not going to
                   fret over this at the moment. */
                //log("seen auxdata",localAux);
                T.assert(pAux===localAux);
              }
              return x;
            }
          });
          db.exec([
            "create table t(a);",
            "insert into t(a) values(1),(2),(3);",
            "select auxtest(a,a), auxtest(a,a) from t order by a"
          ]);
        }finally{
          db.close();
          wasm.pstack.restore(stack);
        }
        T.assert(6===args.length);
        T.assert(pAuxDestructed>0);
        T.assert(pAuxDtorDestructed);
      }
    })
  ;/*end of Bug Reports group*/;

  ////////////////////////////////////////////////////////////////////////
  log("Loading and initializing sqlite3 WASM module...");
  if(0){
    globalThis.sqlite3ApiConfig = {