SQLite User Forum

memory leaks in sqlite3.c:23618
Login

memory leaks in sqlite3.c:23618

(1) By puppet2333 (zxc8263735) on 2020-05-26 07:10:02 updated by 1.1 [link] [source]

Version: SQLite version 3.32.0 2020-04-06 18:16:31

OS: Ubuntu 16.04 LTS

POC: https://github.com/puppet-meteor/NLP_POC/blob/master/sqlite3/POC_6_000667

cmd: ./sqlite3 < POC

ASAN log:

=================================================================
==29374==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 248 byte(s) in 1 object(s) allocated from:

    #0 0x7ffff6f02961 in realloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98961)
    #1 0x4617f3 in sqlite3MemRealloc ../sqlite3.c:23618
    #2 0x46309a in sqlite3Realloc ../sqlite3.c:27652
    #3 0x4631a3 in sqlite3_realloc64 ../sqlite3.c:27684
    #4 0x469423 in sqlite3StrAccumEnlarge ../sqlite3.c:28918
    #5 0x4697cc in enlargeAndAppend ../sqlite3.c:28955
    #6 0x46991a in sqlite3_str_append ../sqlite3.c:28972
    #7 0x464d01 in sqlite3_str_vappendf ../sqlite3.c:28249
    #8 0x46a7d6 in sqlite3_vmprintf ../sqlite3.c:29157
    #9 0x46a9af in sqlite3_mprintf ../sqlite3.c:29173
    #10 0x448947 in do_meta_command ../shell.c:15871
    #11 0x455fb1 in process_input ../shell.c:18527
    #12 0x4588e6 in main ../shell.c:19330
    #13 0x7ffff617c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 248 byte(s) leaked in 1 allocation(s).

memory leaks in sqlite3.c:23618

(1.1) By puppet2333 (zxc8263735) on 2020-05-27 14:24:03 edited from 1.0 [link] [source]

Version: SQLite version 3.32.0 2020-04-06 18:16:31

OS: Ubuntu 16.04 LTS

POC: https://github.com/puppet-meteor/NLP_POC/blob/master/sqlite3/POC_6_000667

cmd: ./sqlite3 < POC

ASAN log:

=================================================================
==29374==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 248 byte(s) in 1 object(s) allocated from:

    #0 0x7ffff6f02961 in realloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98961)
    #1 0x4617f3 in sqlite3MemRealloc ../sqlite3.c:23618
    #2 0x46309a in sqlite3Realloc ../sqlite3.c:27652
    #3 0x4631a3 in sqlite3_realloc64 ../sqlite3.c:27684
    #4 0x469423 in sqlite3StrAccumEnlarge ../sqlite3.c:28918
    #5 0x4697cc in enlargeAndAppend ../sqlite3.c:28955
    #6 0x46991a in sqlite3_str_append ../sqlite3.c:28972
    #7 0x464d01 in sqlite3_str_vappendf ../sqlite3.c:28249
    #8 0x46a7d6 in sqlite3_vmprintf ../sqlite3.c:29157
    #9 0x46a9af in sqlite3_mprintf ../sqlite3.c:29173
    #10 0x448947 in do_meta_command ../shell.c:15871
    #11 0x455fb1 in process_input ../shell.c:18527
    #12 0x4588e6 in main ../shell.c:19330
    #13 0x7ffff617c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 248 byte(s) leaked in 1 allocation(s).

(2) By Larry Brasfield (LarryBrasfield) on 2020-05-26 16:30:42 in reply to 1.0 [link] [source]

It would be more meaningful and helpful if you could replicate this leak detection with the actual 3.32.0 release code, published 2020-05-22.

Also, I took a look at line 15871 of the prerelease shell.c version nearest to the 2020-04-06 date of the code you apparently used, and could see that the memory allocated by the call to sqlite3_mprintf() is in fact reliably freed later in the same sequence of statements, without the possibility of branches avoiding the free. For that reason, it would be useful to know what input was being processed to produce the alleged leak.

(3) By puppet2333 (zxc8263735) on 2020-05-27 06:03:49 in reply to 2 [link] [source]

Hi, the the ASAN logs on the state-of-the-art version 3.32.1 2020-05-25 16:19:56 are as follows:

==40745==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 240 byte(s) in 1 object(s) allocated from:

#0 0x7ffff6f02961 in realloc (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x98961)
#1 0x4628f7 in sqlite3MemRealloc /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:23697
#2 0x46416c in sqlite3Realloc /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:27744
#3 0x46a494 in sqlite3StrAccumEnlarge /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:29026
#4 0x46a83d in enlargeAndAppend /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:29063
#5 0x46a98b in sqlite3_str_append /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:29080
#6 0x465d5c in sqlite3_str_vappendf /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:28350
#7 0x46b847 in sqlite3_vmprintf /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:29265
#8 0x46ba20 in sqlite3_mprintf /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:29281
#9 0x4493a6 in do_meta_command /data3/ASAN2/sqlite-src-3320100/ASAN/shell.c:16014
#10 0x457078 in process_input /data3/ASAN2/sqlite-src-3320100/ASAN/shell.c:18718
#11 0x459a68 in main /data3/ASAN2/sqlite-src-3320100/ASAN/shell.c:19529
#12 0x7ffff648582f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)

SUMMARY: AddressSanitizer: 240 byte(s) leaked in 1 allocation(s).

The relevant codes in sqlite3.c:

static void *sqlite3MemRealloc(void *pPrior, int nByte){
#ifdef SQLITE_MALLOCSIZE
  void *p = SQLITE_REALLOC(pPrior, nByte);                //line 23697
  if( p==0 ){
    testcase( sqlite3GlobalConfig.xLog!=0 );
    sqlite3_log(SQLITE_NOMEM,
      "failed memory resize %u to %u bytes",
      SQLITE_MALLOCSIZE(pPrior), nByte);
  }
  return p;
#else
  sqlite3_int64 *p = (sqlite3_int64*)pPrior;
  assert( pPrior!=0 && nByte>0 );
  assert( nByte==ROUND8(nByte) ); 
  p--;
  p = SQLITE_REALLOC(p, nByte+8 );
  if( p ){
    p[0] = nByte;
    p++;
  }else{
    testcase( sqlite3GlobalConfig.xLog!=0 );
    sqlite3_log(SQLITE_NOMEM,
      "failed memory resize %u to %u bytes",
      sqlite3MemSize(pPrior), nByte);
  }
  return (void*)p;
#endif
}

(4) By anonymous on 2020-05-27 09:08:42 in reply to 3 [link] [source]

#8 0x46ba20 in sqlite3_mprintf /data3/ASAN2/sqlite-src-3320100/ASAN/sqlite3.c:29281
#9 0x4493a6 in do_meta_command /data3/ASAN2/sqlite-src-3320100/ASAN/shell.c:16014

This leak is in the shell, not sqlite3 library itself. Given the blob with dots you are feeding it, I don't think it reaches the sqlite3_prepare_v2() API. If you fed it directly to sqlite3_prepare_v2(), it would return a parse error without leaking memory.

(5) By Larry Brasfield (LarryBrasfield) on 2020-05-28 22:32:45 in reply to 1.1 updated by 5.1 [source]

I downloaded that file you linked. Because it looked like rubbish of some kind, with only sporadic and slight resemblance to anything that might be reasonably input to the SQLite3 shell, I ignored it for days.  But it kept bugging me that code which so clearly reads as correct, (in shell.c in do\_meta\_command() as indicated above), would leak or be tagged as leaking.  Interestingly, that code does a realloc, (also handled correctly), so I wondered if it was a false positive, possibly due to not instrumenting realloc() calls properly.

Curiosity finally prevailed, so, I: instrumented a shell build with leak detection, ([for MSVC via the CRT](https://docs.microsoft.com/en-us/visualstudio/debugger/finding-memory-leaks-using-the-crt-library?view=vs-2019)); tested it by temporarily commenting out a free() call; then ran it under the debugger so that the detection output to the "debug window" could be seen. When I do this and shove that rubbishy POC_6_000667 file into its input, then tell it to exit whereupon leaks are reported, no leaks are reported.

Because the code weakly implicated as being leaky by the OP does not have platform-specific logic related to allocation, I contend that this test shows the LeakSanitizer to either have been misused or be unable to track realloc operations properly.

I intend to now ignore this putative "bug" which seems to depend upon an unstated application of LeakSanitizer.

(5.1) By Larry Brasfield (LarryBrasfield) on 2020-05-28 22:35:33 edited from 5.0 in reply to 1.1 [link] [source]

I downloaded that file you linked. Because it looked like rubbish of some kind, with only sporadic and slight resemblance to anything that might be reasonably input to the SQLite3 shell, I ignored it for days. But it kept bugging me that code which so clearly reads as correct, (in shell.c in do_meta_command() as indicated above), would leak or be tagged as leaking. Interestingly, that code does a realloc, (also handled correctly), so I wondered if it was a false positive, possibly due to not instrumenting realloc() calls properly.

Curiosity finally prevailed, so, I: instrumented a shell build with leak detection, (for MSVC via the CRT); tested it by temporarily commenting out a free() call; then ran it under the debugger so that the detection output to the "debug window" could be seen. When I do this and shove that rubbishy POC_6_000667 file into its input, then tell it to exit whereupon leaks are reported, no leaks are reported.

Because the code weakly implicated as being leaky by the OP does not have platform-specific logic related to allocation, I contend that this test shows the LeakSanitizer to either have been misused or be unable to track realloc operations properly.

I intend to now ignore this putative "bug" which seems to depend upon an unstated application of LeakSanitizer.

(6) By anonymous on 2020-05-29 18:46:07 in reply to 1.1 [link] [source]

Looks like in this edge case, the code is jumping out of "do_meta_command" function without freeing "zLike".

Below is a patch@4e1db8e9a9ee37. This clears the sanitizer trap in my test.

==================================================================
--- src/shell.c.in
+++ src/shell.c.in
@@ -7493,10 +7493,11 @@
         if( strcmp(z,"preserve-rowids")==0 ){
 #ifdef SQLITE_OMIT_VIRTUALTABLE
           raw_printf(stderr, "The --preserve-rowids option is not compatible"
                              " with SQLITE_OMIT_VIRTUALTABLE\n");
           rc = 1;
+          if( zLike ) sqlite3_free(zLike);
           goto meta_command_exit;
 #else
           ShellSetFlag(p, SHFLG_PreserveRowid);
 #endif
         }else
@@ -7504,10 +7505,11 @@
           ShellSetFlag(p, SHFLG_Newlines);
         }else
         {
           raw_printf(stderr, "Unknown option \"%s\" on \".dump\"\n", azArg[i]);
           rc = 1;
+          if( zLike ) sqlite3_free(zLike);
           goto meta_command_exit;
         }
       }else if( zLike ){
         zLike = sqlite3_mprintf("%z OR name LIKE %Q ESCAPE '\\'",
                 zLike, azArg[i]);