Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Add a more complete test for [76c8435a] and add some commentary about (A) the inability to automatically clean up automatically-generated WASM proxy functions for sqlite3_set_auxdata() destructors and (B) how to deal with (A) to avoid leaking WASM proxy functions. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA3-256: |
d693c2dddbd10a2e0b77893b04b11502 |
User & Date: | stephan 2025-02-03 14:55:56.185 |
Context
2025-02-03
| ||
17:34 | Rework [76c8435a] to eliminate automatic JS-to-WASM function conversions of sqlite3_set_auxdata() destructors because it can leads to leaks on every call of a UDF. This feature never worked before [76c8435a] but fixing it was ill-conceived because of the memory leakage it introduces. WASM function pointers can still be used as destructors in this context. (check-in: 3fb993af0c user: stephan tags: trunk) | |
16:26 | Initial work on a fix for the SAHPool VFS's effectively-no-op digest calculation, as reported in ticket #97 of the downstream npm subproject. This requires more testing alongside databases created before this version to ensure that it's backwards-compatible. (check-in: 9234c33f92 user: stephan tags: sahpool-digest) | |
15:17 | Merge the latest trunk changes into the reuse-schema branch. (check-in: 858163f938 user: drh tags: reuse-schema) | |
15:07 | Merge the latest trunk enhancements into the wal2 branch. (check-in: e2d4c1890a user: drh tags: wal2) | |
14:59 | Merge all the latest trunk enhancements and fixes into the begin-concurrent branch. (check-in: f456a72e0c user: drh tags: begin-concurrent) | |
14:55 | Add a more complete test for [76c8435a] and add some commentary about (A) the inability to automatically clean up automatically-generated WASM proxy functions for sqlite3_set_auxdata() destructors and (B) how to deal with (A) to avoid leaking WASM proxy functions. (check-in: d693c2dddb user: stephan tags: trunk) | |
14:44 | Fix the build process on Windows so that it generates identical sqlite3.c, sqlite3.h, and shell.c files on Windows and Unix. This patch also includes a change to JS bindings that got caught up in the branch. (check-in: 91ef45fc29 user: drh tags: trunk) | |
Changes
Changes to ext/wasm/api/sqlite3-api-glue.c-pp.js.
︙ | ︙ | |||
224 225 226 227 228 229 230 231 232 233 234 235 236 237 | new wasm.xWrap.FuncPtrAdapter({ name: 'sqlite3_rollback_hook', signature: 'v(p)', contextKey: (argv)=>argv[0/* sqlite3* */] }), '*' ]], ["sqlite3_set_auxdata", undefined, [ "sqlite3_context*", "int", "*", new wasm.xWrap.FuncPtrAdapter({ name: 'xDestroyAuxData', signature: 'v(p)', contextKey: (argv, argIndex)=>argv[0/* sqlite3_context* */] }) | > > > > > > > > > > > > > > > > > > > > > > > > > | 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 | new wasm.xWrap.FuncPtrAdapter({ name: 'sqlite3_rollback_hook', signature: 'v(p)', contextKey: (argv)=>argv[0/* sqlite3* */] }), '*' ]], /** 2025-02-03: We do not have a way to automatically clean up destructors which are automatically converted from JS functions via the final argument to sqlite3_set_auxdata(). Because of that, it is strongly recommended that clients use wasm.installFunction() to create such callbacks, then pass that pointer to sqlite3_set_auxdata(). Relying on automated conversions here will lead to leaks of JS/WASM proxy functions because sqlite3_set_auxdata() is frequently called in UDFs. The sqlite3.oo1.DB class's onclose handlers can be used for this purpose. For example: const pAuxDtor = wasm.installFunction('v(p)', function(ptr){ //free ptr }); myDb.onclose = { after: ()=>{ wasm.uninstallFunction(pAuxDtor); } }; Then pass pAuxDtor as the final argument to appropriate sqlite3_set_auxdata() calls. */ ["sqlite3_set_auxdata", undefined, [ "sqlite3_context*", "int", "*", new wasm.xWrap.FuncPtrAdapter({ name: 'xDestroyAuxData', signature: 'v(p)', contextKey: (argv, argIndex)=>argv[0/* sqlite3_context* */] }) |
︙ | ︙ | |||
1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 | 'sqlite3_commit_hook', 'sqlite3_preupdate_hook', 'sqlite3_progress_handler', 'sqlite3_rollback_hook', 'sqlite3_set_authorizer', 'sqlite3_trace_v2', 'sqlite3_update_hook' ]) { const x = wasm.exports[name]; if( !x ){ /* assume it was built without this API */ continue; } closeArgs.length = x.length/*==argument count*/ | > > > > | 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 | 'sqlite3_commit_hook', 'sqlite3_preupdate_hook', 'sqlite3_progress_handler', 'sqlite3_rollback_hook', 'sqlite3_set_authorizer', 'sqlite3_trace_v2', 'sqlite3_update_hook' /* We do not yet have a way to clean up automatically-converted sqlite3_set_auxdata() finalizers. */ ]) { const x = wasm.exports[name]; if( !x ){ /* assume it was built without this API */ continue; } closeArgs.length = x.length/*==argument count*/ |
︙ | ︙ |
Changes to ext/wasm/tester1.c-pp.js.
︙ | ︙ | |||
3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 | tryOne(false, "Emscripten filesystem"); tryOne(poolConfig.name); tryOne('opfs'); }finally{ await poolUtil.removeVfs(); } } }) ;/*end of Bug Reports group*/; //////////////////////////////////////////////////////////////////////// log("Loading and initializing sqlite3 WASM module..."); if(0){ globalThis.sqlite3ApiConfig = { | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 3446 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 3494 3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 3506 3507 3508 3509 3510 3511 3512 | tryOne(false, "Emscripten filesystem"); tryOne(poolConfig.name); tryOne('opfs'); }finally{ await poolUtil.removeVfs(); } } }) .t({ /* https://github.com/sqlite/sqlite-wasm/issues/92 */ name: 'sqlite3_set_auxdata() binding signature', test: function(sqlite3){ const db = new sqlite3.oo1.DB(); const stack = wasm.pstack.pointer; const pAux = wasm.pstack.alloc(4); let pAuxDestructed = 0; const args = []; const pAuxDtor = wasm.installFunction('v(p)', function(ptr){ //log("freeing auxdata"); ++pAuxDestructed; }); let pAuxDtorDestructed = false; db.onclose = { after: ()=>{ pAuxDtorDestructed = true; wasm.uninstallFunction(pAuxDtor); } }; try{ db.createFunction("auxtest",{ xFunc: function(pCx, x, y){ args.push(x); T.assert(wasm.isPtr(pCx)); const localAux = capi.sqlite3_get_auxdata(pCx, 0); if( !localAux ){ //log("setting auxdata"); /** We do not currently an automated way to clean up auxdata finalizer functions (the 4th argument to sqlite3_set_auxdata()) which get automatically converted from JS to WASM. Because of that, relying on automated conversions for those is not recommended. Instead, follow the pattern show in this function: use wasm.installFunction() to create the function, then pass the resulting function pointer this function, and cleanup (at some point) using wasm.uninstallFunction(). */ capi.sqlite3_set_auxdata(pCx, 0, pAux, pAuxDtor); }else{ /* This is never actually hit in this example and it's not entirely clear how to cause it to. The point of this test, however, is to demonstrate that the finalizer impl gets triggered, so we're not going to fret over this at the moment. */ //log("seen auxdata",localAux); T.assert(pAux===localAux); } return x; } }); db.exec([ "create table t(a);", "insert into t(a) values(1),(2),(3);", "select auxtest(a,a), auxtest(a,a) from t order by a" ]); }finally{ db.close(); wasm.pstack.restore(stack); } T.assert(6===args.length); T.assert(pAuxDestructed>0); T.assert(pAuxDtorDestructed); } }) ;/*end of Bug Reports group*/; //////////////////////////////////////////////////////////////////////// log("Loading and initializing sqlite3 WASM module..."); if(0){ globalThis.sqlite3ApiConfig = { |
︙ | ︙ |