parallel buils broken

(1) By anonymous on 2022-06-26 10:26:36 [link] [source]

Arch Linux parallel build fails for some time here unless one disables smp build:

config.status: executing libtool commands
gcc  -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -o mksourceid /build/sqlite/src/sqlite-src-3390000/tool/mksourceid.c
gcc  -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -o mkkeywordhash -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_DBSTAT_VTAB=1 -DSQLITE_ENABLE_FTS3_TOKENIZER=1 -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_MATH_FUNCTIONS  /build/sqlite/src/sqlite-src-3390000/tool/mkkeywordhash.c
gcc  -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -o lemon /build/sqlite/src/sqlite-src-3390000/tool/lemon.c
./libtool --mode=compile --tag=CC gcc -DSQLITE_SECURE_DELETE -DSQLITE_MAX_VARIABLE_NUMBER=250000 -DSQLITE_MAX_EXPR_DEPTH=10000  -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -DSQLITE_OS_UNIX=1 -I. -I/build/sqlite/src/sqlite-src-3390000/src -I/build/sqlite/src/sqlite-src-3390000/ext/rtree -I/build/sqlite/src/sqlite-src-3390000/ext/icu -I/build/sqlite/src/sqlite-src-3390000/ext/fts3 -I/build/sqlite/src/sqlite-src-3390000/ext/async -I/build/sqlite/src/sqlite-src-3390000/ext/session -I/build/sqlite/src/sqlite-src-3390000/ext/userauth -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -DNDEBUG -I/usr/include -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_DBSTAT_VTAB=1 -DSQLITE_ENABLE_FTS3_TOKENIZER=1 -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_MATH_FUNCTIONS  -DSQLITE_HAVE_ZLIB=1  -DSQLITE_TEMP_STORE=1 -c /build/sqlite/src/sqlite-src-3390000/src/json.c
./libtool --mode=compile --tag=CC gcc -DSQLITE_SECURE_DELETE -DSQLITE_MAX_VARIABLE_NUMBER=250000 -DSQLITE_MAX_EXPR_DEPTH=10000  -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -DSQLITE_OS_UNIX=1 -I. -I/build/sqlite/src/sqlite-src-3390000/src -I/build/sqlite/src/sqlite-src-3390000/ext/rtree -I/build/sqlite/src/sqlite-src-3390000/ext/icu -I/build/sqlite/src/sqlite-src-3390000/ext/fts3 -I/build/sqlite/src/sqlite-src-3390000/ext/async -I/build/sqlite/src/sqlite-src-3390000/ext/session -I/build/sqlite/src/sqlite-src-3390000/ext/userauth -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -DNDEBUG -I/usr/include -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_DBSTAT_VTAB=1 -DSQLITE_ENABLE_FTS3_TOKENIZER=1 -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_MATH_FUNCTIONS  -DSQLITE_HAVE_ZLIB=1  -DSQLITE_CORE -c /build/sqlite/src/sqlite-src-3390000/ext/misc/stmt.c
tclsh8.6 /build/sqlite/src/sqlite-src-3390000/tool/mkshellc.tcl >shell.c
libtool: compile:  gcc -DSQLITE_SECURE_DELETE -DSQLITE_MAX_VARIABLE_NUMBER=250000 -DSQLITE_MAX_EXPR_DEPTH=10000 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -DSQLITE_OS_UNIX=1 -I. -I/build/sqlite/src/sqlite-src-3390000/src -I/build/sqlite/src/sqlite-src-3390000/ext/rtree -I/build/sqlite/src/sqlite-src-3390000/ext/icu -I/build/sqlite/src/sqlite-src-3390000/ext/fts3 -I/build/sqlite/src/sqlite-src-3390000/ext/async -I/build/sqlite/src/sqlite-src-3390000/ext/session -I/build/sqlite/src/sqlite-src-3390000/ext/userauth -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -DNDEBUG -I/usr/include -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_DBSTAT_VTAB=1 -DSQLITE_ENABLE_FTS3_TOKENIZER=1 -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_HAVE_ZLIB=1 -DSQLITE_TEMP_STORE=1 -c /build/sqlite/src/sqlite-src-3390000/src/json.c  -fPIC -DPIC -o .libs/json.o
libtool: compile:  gcc -DSQLITE_SECURE_DELETE -DSQLITE_MAX_VARIABLE_NUMBER=250000 -DSQLITE_MAX_EXPR_DEPTH=10000 -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -flto=auto -DSQLITE_OS_UNIX=1 -I. -I/build/sqlite/src/sqlite-src-3390000/src -I/build/sqlite/src/sqlite-src-3390000/ext/rtree -I/build/sqlite/src/sqlite-src-3390000/ext/icu -I/build/sqlite/src/sqlite-src-3390000/ext/fts3 -I/build/sqlite/src/sqlite-src-3390000/ext/async -I/build/sqlite/src/sqlite-src-3390000/ext/session -I/build/sqlite/src/sqlite-src-3390000/ext/userauth -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -DNDEBUG -I/usr/include -DSQLITE_THREADSAFE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_DBSTAT_VTAB=1 -DSQLITE_ENABLE_FTS3_TOKENIZER=1 -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_HAVE_ZLIB=1 -DSQLITE_CORE -c /build/sqlite/src/sqlite-src-3390000/ext/misc/stmt.c  -fPIC -DPIC -o .libs/stmt.o
In file included from /build/sqlite/src/sqlite-src-3390000/src/json.c:25:
/build/sqlite/src/sqlite-src-3390000/src/sqliteInt.h:595:10: fatal error: parse.h: No such file or directory
  595 | #include "parse.h"
      |          ^~~~~~~~~
compilation terminated.
make: *** [Makefile:878: json.lo] Error 1

Works well though when -j1 is applied.

(2) By Richard Hipp (drh) on 2022-06-26 10:43:33 in reply to 1 [link] [source]

What sequence of commands did you enter in order to get this error?

(3) By andyrtr on 2022-06-26 11:34:27 in reply to 2 [link] [source]

Arch Linux PKGBUILD

The build only succeeds when applying !makeflags option that sets -j1 jobs.

(4.1) By Richard Hipp (drh) on 2022-06-26 15:32:25 edited from 4.0 in reply to 3 [link] [source]

Deleted

(5) By Richard Hipp (drh) on 2022-06-27 12:25:50 in reply to 1 [source]

Fixed on trunk. The fix will be in the next release.

Note that this problem only comes up when using the --disable-amalgamation option on ./configure. Note also that using the --disable-amalgamation option causes SQLite turn run between 5% and 10% slower. You could reduce that slowdown somewhat by manually adding the -flto option to the compiler, but the resulting library is still about 2% slower than a normal build. For the best possible performance, you should omit --disable-amalgamation.

The developers never use --disable-amalgamation, which is why this problem has gone unnoticed.

(6) By andyrtr on 2022-06-27 19:41:29 in reply to 5 [link] [source]

I can confirm the commit in trunk indeed fixes the issue here.

(And we follow your suggestion to use amalgamation)