SQLite Forum

Inconsistent memory stats of sqlite3 with different compilers

Inconsistent memory stats of sqlite3 with different compilers

(1) By Shaohua (shaohua.li) on 2021-06-10 08:53:31 [link] [source]

Hi there,

I found that sqlite3 -stats compiled with gcc11 and clang12 would emit different memory stats on some crafted inputs.

  • SQLite version: 3.36.0

  • Compiler args: -DSQLITE_MAX_LENGTH=128000000 \ -DSQLITE_MAX_SQL_LENGTH=128000000 \ -DSQLITE_MAX_MEMORY=25000000 \ -DSQLITE_PRINTF_PRECISION_LIMIT=1048576 \ -DSQLITE_DEBUG=1 \ -DSQLITE_MAX_PAGE_COUNT=16384 (I used these args for avoiding creating large databases and irrelevant OOMs)

  • Compilers: gcc11 and clang12

  • Reproduce: run the following command with different compilers compiled binaries. Then compare the two stats.txt.

    sqlite3 -stats < diff.sql | xargs -i echo {} > stats.txt

You can download diff.sql from this link: diff.sql

(2) By Richard Hipp (drh) on 2021-06-10 14:51:42 in reply to 1 [source]

The "diff.sql" is fuzzer output. What are you doing? Are you working on something useful, or are you just trying to break things?

(3.3) By Simon Slavin (slavin) on 2021-06-12 09:38:58 edited from 3.2 in reply to 1 [link] [source]

Small differences are a common result of how the compilers allocate memory and optimise memory use. Small differences in memory-use between compilers, even between different versions of the same compiler, do not indicate any fault.

You are especially likely to see this using -DSSQLITE_DEBUG=1 because it includes many assertions and the compiler doesn't expect to see assert in production code, so it doesn't bother optimising the code around it.

If one difference is more than 1,024 bytes, you might have found a bug in one of the compilers.

If you want to explore these differences, you might set compiler options to turn optimisation off on both of them, and see whether there's still a difference in the results they generate.