When attempting to generate a changeset of a sqlite3_session attached to table(s) which have tracked >= 1.0 GB of data, sqlite3session_changeset fails with SQLITE_NOMEM. Similarly when attempting to generate a changeset of a sqlite3_session from a sqlite3session_diff which is >= 1.0 GB of data, sqlite3session_changeset fails with SQLITE_NOMEM. When looking at the documentation for the session extension: https://sqlite.org/sessionintro.html and sqlite3session_changeset: https://sqlite.org/session/sqlite3session_changeset.html I don't see any mention of this limit/restriction. The culprit appears to be sessionBufferGrow(SessionBuffer, size_t, int): Specifically this block of code: ``` i64 nNew = p->nAlloc ? p->nAlloc : 128; do { nNew = nNew*2; }while( (size_t)(nNew-p->nBuf)<nByte ); ``` If p->nAlloc >= 1073741696 which is then set to nNew it doubles once and will be >= 2147483392 which is over the max threshold in sqlite3Realloc (see below). Eventually this nNew gets passed into sqlite3_realloc64(void *, sqlite3_uint64) which calls through the sqlite3Realloc(void *, u64) where we hit our limit at this block of code: ``` if( nBytes>=0x7fffff00 ){ /* The 0x7ffff00 limit term is explained in comments on sqlite3Malloc() */ return 0; } ``` The comment in sqlite3Malloc() is as follows: ``` /* A memory allocation of a number of bytes which is near the maximum ** signed integer value might cause an integer overflow inside of the ** xMalloc(). Hence we limit the maximum size to 0x7fffff00, giving ** 255 bytes of overhead. SQLite itself will never use anything near ** this amount. The only way to reach the limit is with sqlite3_malloc() */ ``` My suggested patch to resolve this allows sqlite3session_changeset to successfully generate changesets from sessions tracking >= 1.0 GB and properly fails with SQLITE_NOMEM when attempting to generate changesets from sessions tracking >= 0x7FFFFEFE (2 bytes below the max threshold in sqlite3Malloc()): ``` static int sessionBufferGrow(SessionBuffer *p, size_t nByte, int *pRc){ if( *pRc==SQLITE_OK ){ if( (size_t)(p->nAlloc-p->nBuf)<nByte ){ u8 *aNew; i64 nNew = p->nAlloc ? p->nAlloc : 128; do { /* limit growth to 2 bytes below our max threshold as defined in sqlite3Malloc(), otherwise we would hit OOM condition when requesting to grow the buffer >= 1/2 max threshold */ nNew = MIN(nNew*2, 0x7FFFFEFE); /* if we've hit 2 bytes below our max threshold and still haven't reached the requested size, break out with OOM */ if ( nNew == 0x7FFFFEFE && (size_t)(nNew-p->nBuf)<nByte ){ *pRc = SQLITE_NOMEM; break; } }while( (size_t)(nNew-p->nBuf)<nByte ); aNew = (u8 *)sqlite3_realloc64(p->aBuf, nNew); if( 0==aNew ){ *pRc = SQLITE_NOMEM; }else{ p->aBuf = aNew; p->nAlloc = nNew; } } } return (*pRc!=SQLITE_OK); } ``` I've linked a reproduction case displaying the issue when attempting to generate a chanegset using sqlite3session_changeset from a session tracking tables and an additional example when the session was from a sqlite3session_diff: https://gist.github.com/R4N/f12407697e06bd22f9111cb9f4eb4a16