Caution: clang-11.0.0 miscompiles SQLite
(1.2) By Richard Hipp (drh) on 2020-06-04 12:20:24 edited from 1.1 [source]
It appears that the clang-11.0.0 compiler mis-compiles sqlite3.c version 3.32.1. I have checked in a change to SQLite that appears to work around the problem. But there might be other bugs. Therefore, use clang-11.0.0 with caution and test your applications carefully!
UPDATE: I learn (from a thread on HN) that the compiler in question is pre-release.
UPDATE-2: The LLVM devs apparently saw this post on HN and quickly isolated the problem. They have a ticket. Great work, y'all!
Details:
OSSFuzz has been reporting bug 23003 against SQLite. I could not reproduce the problem on my desktop (Ubuntu with gcc-5.4.0) so I followed the OSSFuzz bug replication procedures and discovered what appears to be a problem with Clang-11.0.0 currently used by OSSFuzz.
The code that is miscompiled is lines 345-347 of the src/utf.c source file, shown below:
c = pMem->flags;
sqlite3VdbeMemRelease(pMem);
pMem->flags = MEM_Str|MEM_Term|(c&(MEM_AffMask|MEM_Subtype));
From the -S output, it looks like Clang-11.0.0 is compiling these three lines as if there were written as:
sqlite3VdbeMemRelease(pMem);
pMem->flags = MEM_Str|MEM_Term|(pMem->flags&(MEM_AffMask|MEM_Subtype));
In other words, Clang seems to be assuming that the sqlite3VdbeMemRelease() function does not change the value of pMem->flags. But it does. My work-around is to do the bit-twiddling of pMem->flags before the function call instead of afterwards:
c = MEM_Str|MEM_Term|(pMem->flags&(MEM_AffMask|MEM_Subtype));
sqlite3VdbeMemRelease(pMem);
pMem->flags = c;
Compiler Version And Build Details:
OSSFuzz reports the compiler used as:
clang version 11.0.0 (https://github.com/llvm/llvm-project.git a6ae333a0c23fc9b0783ca45e2676abac00c6723)
Target: x86_64-unknown-linux-gnu
Thread model: posix
The build script compiles SQLite thusly:
clang -O1 -fno-omit-frame-pointer -gline-tables-only -DFUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION -fsanitize=address -fsanitize-address-use-after-scope -fsanitize=fuzzer-no-link -DSQLITE_MAX_LENGTH=128000000 -DSQLITE_MAX_SQL_LENGTH=128000000 -DSQLITE_MAX_MEMORY=25000000 -DSQLITE_PRINTF_PRECISION_LIMIT=1048576 -DSQLITE_DEBUG=1 -DSQLITE_MAX_PAGE_COUNT=16384 -O1 -g -I. -c -O1 -g ./sqlite3.c -o sqlite3.o