The RSQLite R package Address Sanitizer warning
(1) By Daniel E. Weeks (dweeks) on 2025-06-26 13:24:36 [source]
Automated checks by CRAN of our Mega2R R package generated this warning:
Flavor: r-devel-linux-x86_64-debian-special-gcc-san
Check: Post-processing issues found for gcc-san, Result: WARNING
File: build_vignettes.log
vendor/sqlite3/sqlite3.c:80239:14: runtime error: load of address
0x7faa31b1fa40 with insufficient space for an object of type 'struct
MemPage *'
I asked about this on R-package-devel and they suggested I contact you, saying it might be related to recent changes discussed here:
https://sqlite.org/forum/forumpost/311dbf9a1cadfae6
Further investigations via R-Hub2
https://github.com/r-hub2/diplomatic-mole-Mega2R/actions
find that none of the checks on these machines reproduced the Warning seen on the CRAN gcc-san test machine:
clang-asan: All CRAN checks passed.
clang-ubsan: All CRAN checks passed.
valgrind: All CRAN checks passed.
m1-san (R-devel): The Warning seen on the CRAN gcc-san test machine was not observed. However, there was an error involving seqlevels<-
. This is because the development version of GenomeInfoDb
no longer contains the seqlevels<-
function, which in the next release will be moved into the Seqinfo package.
The latest version of Mega2R is available here
https://bitbucket.org/dweeks/mega2r/src/master/
Thank you,
Dan Weeks
(2) By Richard Hipp (drh) on 2025-06-26 14:11:41 in reply to 1 [link] [source]
What (exact) version is the sqlite3.c file that is reporting the runtime error? You can find this using a command like:
grep 'define SQLITE_SOURCE_ID' sqlite3.c
(3) By Daniel E. Weeks (dweeks) on 2025-06-26 15:28:38 in reply to 2 [link] [source]
Unfortunately, the testing was done automatically on a CRAN machine that I do not have access to.
But in their 'outputs.txt' file, which is available here (but which probably will disappear soon)
I see this:
* installing *source* package ‘RSQLite’ ...
** this is package ‘RSQLite’ version ‘2.4.1’
** package ‘RSQLite’ successfully unpacked and MD5 sums checked
So if I download the source for ‘RSQLite’ version ‘2.4.1’ from CRAN, then I get
$ grep 'define SQLITE_SOURCE_ID' sqlite3.c
#define SQLITE_SOURCE_ID "2025-06-06 14:52:32 b77dc5e0f596d2140d9ac682b2893ff65d3a4140aa86067a3efebe29dc914c95"
(4) By Roger Binns (rogerbinns) on 2025-06-26 16:09:11 in reply to 2 [link] [source]
I've seen the same issue recently. Using current fossil (2025-06-25 20:42:40 cb4d05633a0c9cdf146f3108e1b4b10754cd79d72a425d8cc9cd21836037a01b
) and earlier versions. Amalgamation produced with ./configure --quiet --all --disable-tcl
Operating system is Ubuntu 25.04 and you have to install gcc-15
and compile with optimisation on - -Og
means the issue isn't shown. The compilation needs to use the undefined behaviour sanitizer. Neither gcc-14 or clang show the issue.
sqlite3/sqlite3.c:80356:14: runtime error: load of address 0x7ab294d2ce00 with insufficient space for an object of type 'struct MemPage *'
0x7ab294d2ce00: note: pointer points here
cd cd cd cd 90 0d ed 99 c2 7c 00 00 90 81 ec 99 c2 7c 00 00 cd cd cd cd cd cd cd cd cd cd cd cd
^
Line 80356 corresponds to MemPage *pOld = ...
:
if( (pageFlags & PTF_LEAF)==0 && nOld!=nNew ){
MemPage *pOld = (nNew>nOld ? apNew : apOld)[nOld-1];
memcpy(&apNew[nNew-1]->aData[8], &pOld->aData[8], 4);
}
Relevant portion of backtrace:
#4 0x00007bffe4385eea in balance_nonroot (pParent=pParent@entry=0x7f1ff5b29880, iParentIdx=iParentIdx@entry=0, aOvflSpace=aOvflSpace@entry=0x7e0ff5e28508 '\276' <repeats 200 times>..., isRoot=isRoot@entry=1, bBulk=<optimised out>) at /space/apsw/sqlite3/sqlite3.c:80356
#5 0x00007bffe43898bd in balance (pCur=pCur@entry=0x7edff5aeebb8) at /space/apsw/sqlite3/sqlite3.c:80817
#6 0x00007bffe438ec10 in sqlite3BtreeInsert (pCur=0x7edff5aeebb8, pX=pX@entry=0x7bfff4a384f0, flags=flags@entry=0, seekResult=seekResult@entry=-1) at /space/apsw/sqlite3/sqlite3.c:81272
#7 0x00007bffe44c2262 in sqlite3VdbeExec (p=p@entry=0x7edff5af0748) at /space/apsw/sqlite3/sqlite3.c:101006
#8 0x00007bffe44d2fe7 in sqlite3Step (p=p@entry=0x7edff5af0748) at /space/apsw/sqlite3/sqlite3.c:92408
#9 0x00007bffe44d3831 in sqlite3_step (pStmt=pStmt@entry=0x7edff5af0748) at /space/apsw/sqlite3/sqlite3.c:92470
To cause the report, the code does this:
- Opening file database
- pragma cache_size = 10
- pragma temp_store = 1
- create temp table testissue506(x UNIQUE,y UNIQUE, PRIMARY KEY(x,y));
- Inserting ("", ""), ("a", "b"), ("aa", "bb"), ("aaa", "bbb") etc to spill from file to temp.
Because of the requirements to reproduce the issue, it looks like a compiler issue not a bug in SQLite which is why I didn't report it.
(5) By Richard Hipp (drh) on 2025-06-26 17:01:47 in reply to 4 [link] [source]
I don't have Ubuntu 25 easily at hand. (My machines are on 24.04). And the latest GCC I have is 13.3.0. So I'm not able to repro the problem as originally stated. I did make (an uncommitted) debug modification as follows:
--- src/btree.c +++ src/btree.c @@ -8695,8 +8695,9 @@ /* If the sibling pages are not leaves, ensure that the right-child pointer ** of the right-most new sibling page is set to the value that was ** originally in the same field of the right-most old sibling page. */ if( (pageFlags & PTF_LEAF)==0 && nOld!=nNew ){ + printf("nNew=%d, nOld=%d\n", nNew, nOld); fflush(stdout); MemPage *pOld = (nNew>nOld ? apNew : apOld)[nOld-1]; memcpy(&apNew[nNew-1]->aData[8], &pOld->aData[8], 4); }
I was not able to reach the debug printf() using Roger's recipe. This code is in the b-tree layer which is independent of the page cache, so the "cache_size" and "temp_store" and "spill from file to temp" parts of Roger's recipe seem like red herrings to me, but I don't know. I was able to reach the debugging printf() using this script:
CREATE TABLE t1(x UNIQUE,y UNIQUE,z BLOB,PRIMARY KEY(x,y)); WITH RECURSIVE c(n) AS (VALUES(1) UNION ALL SELECT n+1 FROM c WHERE n<26*20) INSERT INTO t1 SELECT format('%c%.*c',char(0x61+n%26),n,'a'), format('%c%.*c',char(0x61+(n*17)%26),n,'b'), zeroblob(15+n%30) FROM c; DELETE FROM t1 WHERE rowid%19;
Roger: If you insert a temporary printf() has shown above, can you still hit the UBSAN warning? If so, what values of nNew and nOld are shown?
Anybody: Are you able to hit a UBSAN warning by compiling "sqlite3" and running the script above?
(6) By Roger Binns (rogerbinns) on 2025-06-26 17:42:07 in reply to 5 [link] [source]
You can use distrobox to get container environments of other Linux versions, although it may be more hassle than it is worth.
The cache size and temp store pragmas are there because my test case is ensuring my VFS code correctly encounters and handles a NULL filename, ensuring that memory is filled with data and then temp db spill to disk happens. Commenting them out still causes the UBSAN warning.
nNew=2, nOld=1
Your recursive query also reproduces the issue. A CREATE
followed by INSERT
or a large blob does not.
(8) By Roger Binns (rogerbinns) on 2025-06-26 17:53:01 in reply to 6 [link] [source]
I managed to hit it with gcc 14 too. The recursive query is in a file named test.sql
.
At least -O1
must be present.
$ gcc -O1 -fsanitize=undefined -o sqlite3 shell.c sqlite3.c -lm && ./sqlite3 "" ".read test.sql"
nNew=2, nOld=1
sqlite3.c:80357:14: runtime error: load of address 0x7ffd8e63d8e0 with insufficient space for an object of type 'struct MemPage *'
0x7ffd8e63d8e0: note: pointer points here
78 7a 00 00 d0 55 3b f6 6f 59 00 00 e0 87 3b f6 6f 59 00 00 48 45 3b f6 6f 59 00 00 48 5a 8b 3e
^
nNew=3, nOld=2
nNew=4, nOld=3
nNew=2, nOld=1
nNew=2, nOld=1
nNew=4, nOld=3
...
It does not reproduce with clang.
(9) By Richard Hipp (drh) on 2025-06-26 19:04:53 in reply to 8 [link] [source]
The error appears in the final INSERT statement of the script shown at the bottom of this post when compiling and run as follows:
gcc -O1 -g -fsanitize=undefined -o sqlite3 shell.c sqlite3.c -lm -ldl -lpthread ./sqlite3 <ub1.txt
The change at check-in 2025-06-26T18:57Z clears the warning. I agree with Roger: this seems like a gcc/ubsan bug. I'll see if I can enter a bug report that they can use.
FWIW, Dan discovered that the same error comes up with gcc-13. It was not necessary for me to spend all that time getting gcc-14 installed.....
Here is the "ub1.txt" script:
CREATE TABLE t1(x UNIQUE,y UNIQUE,z BLOB,PRIMARY KEY(x,y)); WITH RECURSIVE c(n) AS (VALUES(1) UNION ALL SELECT n+1 FROM c WHERE n<233) INSERT INTO t1 SELECT format('%c%.*c',char(0x61+n%26),n,'a'), format('%c%.*c',char(0x61+(n*17)%26),n,'b'), zeroblob(15+n%30) FROM c; INSERT INTO t1 VALUES( format('a%.*c',234,'a'), format('a%.*c',234,'b'), zeroblob(39) );
(10) By Roger Binns (rogerbinns) on 2025-06-26 19:13:57 in reply to 9 [link] [source]
The change at check-in 2025-06-26T18:57Z ...
That change also makes the warning go away in gcc-15.
It was also occurring with -Os
, so it looked like any optimization level triggered whatever concerned gcc.
(11) By Richard Hipp (drh) on 2025-06-26 19:29:42 in reply to 10 [link] [source]
I filed a bug with GCC at https://gcc.gnu.org/bugzilla/show_bug.cgi?id=120837. We'll see if we get pushback...
(7) By Richard Hipp (drh) on 2025-06-26 17:42:45 in reply to 1 [link] [source]
I asked about this on R-package-devel and they suggested I contact you, saying it might be related to recent changes discussed here: https://sqlite.org/forum/forumpost/311dbf9a1cadfae6
The (extensive) changes to get -fsanitize=strict-bounds to hush up did not affect the MemPage object, which is the focus of this new warning. So that theory seems unlikely.