Segmentation fault in sqlite3_complete function

(1) By Song Liu (songliu) on 2023-03-25 00:23:34 [link] [source]

I found a segment fault while SQLite (latest, 74d14900afd10d02) executes the poc.

The poc is mutated by fuzzing, therefore it contains some special characters and is not good for humans to read.

I upload the poc to google drive, here is the link: poc

the MD5 digest of poc: 161fb28eec736e49afb49f57b2f96c02

Here are the outputs:

91000
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
[1]    867138 segmentation fault  ./sqlite3 < ./poc

Here is the backtrace printed by gdb:

#0  0x00005555556da6da in sqlite3_complete (zSql=0x0) at sqlite3.c:173766
#1  0x000055555558a106 in printSchemaLine (out=0x7ffff7dba6a0 <_IO_2_1_stdout_>, z=0x5555558032b0 "....", zTail=0x55555575bebb ";\n") at shell.c:17241
#2  0x00005555555920d8 in dump_callback (pArg=0x7fffffffc710, nArg=0x3, azArg=0x5555557fc3f8, azNotUsed=0x5555557fc3e0) at shell.c:19545
#3  0x000055555567a6f1 in sqlite3_exec (db=0x5555557eaa20, zSql=0x5555557fff60 "SELECT name, type, sql FROM sqlite_schema AS o WHERE (true) AND type=='table'  AND sql NOT NULL ORDER BY tbl_name='sqlite_sequence', rowid", xCallback=0x555555591e56 <dump_callback>, pArg=0x7fffffffc710, pzErrMsg=0x7fffffffa8f8) at sqlite3.c:132003
#4  0x000055555559250b in run_schema_dump_query (p=0x7fffffffc710, zQuery=0x5555557fff60 "SELECT name, type, sql FROM sqlite_schema AS o WHERE (true) AND type=='table'  AND sql NOT NULL ORDER BY tbl_name='sqlite_sequence', rowid") at shell.c:19630
#5  0x000055555559b1a9 in do_meta_command (zLine=0x5555557ef5c0 ".d", p=0x7fffffffc710) at shell.c:23292
#6  0x00005555555a61a3 in process_input (p=0x7fffffffc710) at shell.c:26437
#7  0x00005555555a865f in main (argc=0x1, argv=0x7fffffffdd38) at shell.c:27363

Note that this crash seems not very stable. For example, patch 1ef461aa4e95d254 is not related to this bug, however, it indeed makes some of my PoCs fail to reproduce the crash. There are also other factors like instrumenting SQLite with ASAN, which may also lead to failed reproduction.

Below are my compilation steps.

$ mkdir sqlite && cd sqlite
$ fossil open ~/Fossils/sqlite.fossil
$ fossil checkout 74d14900afd10d02
$ export CFLAGS="-g -O0 -DSQLITE_DEBUG 
                -DSQLITE_ENABLE_TREETRACE 
                -DSQLITE_ENABLE_WHERETRACE
                -DSQLITE_ENABLE_CURSOR_HINTS 
                -DSQLITE_COUNTOFVIEW_OPTIMIZATION 
                -DSQLITE_ENABLE_STAT4" 
$ ./configure --enable-all --disable-shared && make

(2.1) By Song Liu (songliu) on 2023-03-25 01:07:15 edited from 2.0 in reply to 1 [source]

This bug is rooted in shell.c, and is not related to the core functionality of SQLite.

Following is the related code, while zNew is the return value of sqlite3_mprintf.

When OOM occurs, zNew is NULL and SQLite uses the NULL directly in the sqlite3_complete, which caused a segmentation fault.

17240       char *zNew = sqlite3_mprintf("%s%s;", zOrig, azTerm[i]);
17241       if( sqlite3_complete(zNew) ){
17242         size_t n = strlen(zNew);
17243         zNew[n-1] = 0;
17244         zToFree = zNew;
17245         z = zNew;
17246         break;
17247       }
17248       sqlite3_free(zNew);

I saw SQLite3 use shell_check_oom function to check whether OOM happened. One possible fix is using shell_check_oom function to check zNew.

(3) By Larry Brasfield (larrybr) on 2023-03-25 02:23:29 in reply to 2.1 [link] [source]

Thanks for the bug report. This flaw is fixed now.

What are you doing to expose OOM mis-handling? (I ask as a curious developer.)

(4.1) By Song Liu (songliu) on 2023-03-25 03:17:45 edited from 4.0 in reply to 3 [link] [source]

Thanks for your efforts!

We use greybox fuzzing, a kind of automatic bug finding technique, to discover potential problems hidden in the program. We are developing a new fuzzing tool to test our idea. This case is found by our tool.

(5) By Larry Brasfield (larrybr) on 2023-03-25 03:22:34 in reply to 4.1 [link] [source]

I had surmised the fuzzing part. So the "whitening" of that "greybox" is partly the fuzzer telling a replacement allocator when to fail? Is it departing in other ways from blackbox fuzzing?

(6) By Spindrift (spindrift) on 2023-03-25 16:23:44 in reply to 5 [link] [source]

I suspect the greybox refers to using documentation of your (sqlite's) software implementations to tune the fuzzer, rather than just a black box fuzzer that is ignorant to the inner workings.

(7) By Song Liu (songliu) on 2023-03-25 17:01:46 in reply to 5 [link] [source]

The difference between greybox, whitebox and blackbox in fuzzing is mainly on how sufficient information you have, like in the scenario of blackbox fuzzing, we cannot access the source code and only have stripped binary.

We didn't specialize our tool for testing OOM issues. The interesting fact is that hard_heap_limit plays a key role in finding this bug, while it is automatically and randomly generated by our tool.

You also brought up a good idea for testing OOM issues, it's valuable and feasible. Maybe we can try this method to see how many errors we can catch.