SQLite Forum

GCC build failure with LTO enabled.

GCC build failure with LTO enabled.

(1) By anonymous on 2021-01-19 15:59:58 [link] [source]

Using amalgamation, version : 3.34.0

Gcc version : 9.3.0

../sqlite/sqlite3.c: In function ‘vdbeSorterCompareText’:
../sqlite/sqlite3.c:95150:9: error: ‘memcmp’ specified size 18446744073709551609 exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
95150 |   res = memcmp(v1, v2, (MIN(n1, n2) - 13)/2);
      |         ^
lto1: all warnings being treated as errors
lto-wrapper: fatal error: /usr/bin/cc returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed

We are trying to upgrade Sqlite, to the latest 3.34.0. Build fails with this error when LTO is enabled. Is it familiar to anyone? Maybe a bug at this line? (Hopefully, not a GCC bug.)

Application is quite big, I couldn't trigger this error with a minimal example yet. I'm working on it, I'll post my findings later. Meanwhile, any hint is appreciated.

(2) By Richard Hipp (drh) on 2021-01-19 16:16:10 in reply to 1 [link] [source]

If I'm understanding the error message correctly, GCC is concerned that the size_t expression in the 3rd argument to memcpy() is too large. But that value depends on the values of n1 and n2, which are computed from variable length integers held in memory buffers. It is not reasonable for GCC to compute these values at compile-time. And if it could, it would find that the n1 and n2 values cannot be as large as it claims they are.

So in the absence of further evidence, I'm going to call this a GCC bug.

Further information: I ran:

gcc-9 -flto -c sqlite3.c

and that ran fine for me - no errors. This is with GCC 9.3.0. Do you have an alternative command-line that can replicate the problem?

(3.3) By Keith Medcalf (kmedcalf) on 2021-01-20 04:13:22 edited from 3.2 in reply to 2 [source]

I believe this is a problem with the inliner and it just happens to show up at link time when code generation is occurring at link time. I "used to get" a bunch of error/warning, but presently only get a warning as follows:

sqlite3.c:275788:17: warning: '__builtin_memcmp_eq' specified size between 18446744071562067968 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
275788 |   if( nA==nB && memcmp(zA, zB, nA)==0 ) return 0;
       |                 ^~~~~~~~~~~~~~~~~~

in this function in zipfile.c (which I have added as a builtin extension in sqlite3.c)

** Both (const char*) arguments point to nul-terminated strings. Argument
** nB is the value of strlen(zB). This function returns 0 if the strings are
** identical, ignoring any trailing '/' character in either path.  */
static int zipfileComparePath(const char *zA, const char *zB, int nB){
  int nA = (int)strlen(zA);
  if( nA>0 && zA[nA-1]=='/' ) nA--;
  if( nB>0 && zB[nB-1]=='/' ) nB--;
  if( nA==nB && memcmp(zA, zB, nA)==0 ) return 0;
  return 1;

Testing of the actual code path indicates that this is a bogus warning in that it appears that the warning results from an assumption that nA may be 0 and if it is decremented by 1 then sign extended to 64-bits it will become the number shown, which is an invalid argument to the inline instruction. The compiler is not bright enough to realize that this situation can never actually happen and emits the bogus warning anyway. This warning is fatal when -flto is in effect.

NB: This is with the MinGW64 port of GCC 9.1.0
NB: Same with MinGW64 GCC 10.2.0

(4) By anonymous on 2021-01-21 12:30:47 in reply to 2 [link] [source]

Thank you.

Sorry for the late answer, we were trying to collect more data.

We couldn't isolate the problem to a minimal example unfortunately. For our build, GCC(gcc-10 also behaves same way) decides to inline and generate specific version of that function for a call site(I assume it tries to partially run function on compile time to generate a faster version of it) but when we extract related sources as an another project, it decides not to.

For our case,

  getVarint32NR(&p1[1], n1);
  getVarint32NR(&p2[1], n2);
  res = memcmp(v1, v2, (MIN(n1, n2) - 13)/2);
  if( res==0 ){
    res = n1 - n2;

GCC thinks, MIN(n1, n2) is zero, so it essentially becomes :
memcpy(v1, v2, -6);

Is it possible somehow?

We've disabled that flag for now and we are running our app tests, no issues so far but we'll investigate a bit more and try to get some more debug data from GCC to get actual parameters of the function.

We'll have more info about that in a few days(An ex GCC dev will take a look at GCC). I'll post an update later.

Thank you very much.