In Gentoo when running the test suite using the 3.48.0 version I get some test failures which don't happen with latest 3.47 version.

Configure command is as follows:

./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --libdir=/usr/lib64 --enable-loadextension --enable-threadsafe --enable-fts5 --disable-debug --disable-editline --enable-readline --with-readline-inc=-I/usr/include/readline --disable-static --enable-tcl --with-tcl=/usr/lib64 --soname=legacy

Relevant build log snippet:

e_delete-3.1.1...
Error: near "LIMIT": syntax error
e_delete-3.1.2...
Error: near "LIMIT": syntax error
e_delete-3.1.3...
Error: near "LIMIT": syntax error
e_delete-3.1.4...
Error: near "ORDER": syntax error
e_delete-3.1.5...
Error: near "ORDER": syntax error
e_delete-3.1.6...
Error: near "ORDER": syntax error
e_delete-3.1.7...
Error: near "LIMIT": syntax error
e_delete-3.1.8...
Error: near "LIMIT": syntax error
e_delete-3.1.9...
Error: near "LIMIT": syntax error
e_delete-3.1.10...
Error: near "ORDER": syntax error
e_delete-3.1.11...
Error: near "ORDER": syntax error
e_delete-3.1.12...
Error: near "ORDER": syntax error
./testfixture: near "LIMIT": syntax error
    while executing
"ifcapable update_delete_limit {

# EVIDENCE-OF: R-64942-06615 The LIMIT and ORDER BY clauses (described
# below) are unsupported for DELETE statements..."
    (file "/var/tmp/portage/dev-db/sqlite-3.48.0-r1/work/sqlite-src-3480000-abi_x86_64.amd64/test/e_delete.test" line 274)
    invoked from within
"source /var/tmp/portage/dev-db/sqlite-3.48.0-r1/work/sqlite-src-3480000-abi_x86_64.amd64/test/e_delete.test"
    invoked from within
"interp eval tinterp $script"
    (procedure "slave_test_script" line 30)
    invoked from within
"slave_test_script [list source $zFile] "
    invoked from within
"time { slave_test_script [list source $zFile] }"
    (procedure "slave_test_file" line 23)
    invoked from within
"slave_test_file $file"
    (procedure "run_tests" line 36)
    invoked from within
"run_tests veryquick -presql {} -files {shared3.test /var/tmp/portage/dev-db/sqlite-3.48.0-r1/work/sqlite-src-3480000-abi_x86_64.amd64/test/../ext/rtre..."
    ("uplevel" body line 1)
    invoked from within
"uplevel run_tests $name $::testspec($name)"
    (procedure "run_test_suite" line 5)
    invoked from within
"run_test_suite veryquick"
    (file "/var/tmp/portage/dev-db/sqlite-3.48.0-r1/work/sqlite-src-3480000-abi_x86_64.amd64/test/veryquick.test" line 16)
    invoked from within
"source $argv0"
    invoked from within
"if {[llength $argv]>=1} {
set argv0 [lindex $argv 0]
set argv [lrange $argv 1 end]
source $argv0
} else {
set line {}
while {![eof stdin]} {
if {$line..."
make: *** [/var/tmp/portage/dev-db/sqlite-3.48.0-r1/work/sqlite-src-3480000-abi_x86_64.amd64/main.mk:1693: tcltest] Error 1

You might have some old files in the build directory or something. Those tests should only run if SQLITE_ENABLE_UPDATE_DELETE_LIMIT is defined during the build, and I don't think your configure command does that.

Dan.

I tried running the test suite as part of the Portage package environment both locally and in a new nspawn container, the package manager always cleans the build directory every time the install process is tried so I don't think it's a case of old files in the build directory.

The SQLITE_ENABLE_UPDATE_DELETE_LIMIT argument is being passed, not to configure but rather as a compile-time flag, e.g:

x86_64-pc-linux-gnu-gcc -fPIC -march=native -O2 -pipe  -flto -Werror=odr -Werror=lto-type-mismatch -Werror=str
ict-aliasing  -DSQLITE_ENABLE_API_ARMOR -DSQLITE_ENABLE_BYTECODE_VTAB -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_DBSTAT_VTABDSQLITE_ENABLE_DESERIALIZE -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_HIDDEN_COLUMNS -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_ENABLE_NORMALIZE -DSQLITE_ENABLE_OFFSET_SQL_FUNC -DSQLITE_ENABLE_PREUPDATE_HOOK -DSQLITE_ENABLE_RBU -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_GEOPOLY -DSQLITE_ENABLE_SESSION -DSQLITE_ENABLE_STMT_SCANSTATUS -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT -DSQLITE_SOUNDEX -DSQLITE_USE_URI -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_THREADSAFE=1   -DNDEBUG -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -I. -I/var/tmp/portage/dev-db/sqlite-3.48.0-r1/work/sqlite-src-3480000-abi_x86_64.amd64/src

I can upload the full build log somewhere if needed. I haven't changed any of the build steps in 3.48 compared to 3.47 (apart from making necessary changes to migrate from autoconf to autosetup) but these tests are passing in 3.47, is it possible that it's some kind of regression?

I can upload the full build log somewhere if needed.

That would be helpful, yes. If you'd like to avoid publicizing it, please send it to "support" at this domain.

... is it possible that it's some kind of regression?

It is possible but it sounds like it's more likely a build regression than a library regression.

If you'd like to avoid publicizing it, please send it to "support" at this domain.

It's not a problem, I have uploaded the log here: https://dpaste.com/AYZEZMLKS

I have uploaded the log here:

Thank you! The test failure is reproducible with your list of cflags but not in a vanilla build. A script is currently running to narrow down which cflag(s) trigger(s) it and we'll get back to you when we have a resolution.

A script is currently running to narrow down which cflag(s) trigger(s) it and we'll get back to you when we have a resolution.

This is definitely a build regression which we'll fix today and get into the 3.48 branch. The workaround for your specific case is to modify your "make" invocation:

$ make "OPTIONS=-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT"

Background: the legacy build inspected the CFLAGS for -DSQLITE_... flags and copied all of those into the separate feature-flag list. The 3.48 build keeps those separate, preferring to define the latter solely via configure flags. So what's happening is the lemon parser generator is not seeing the -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT flag, which causes it to generate a parser which does not know about that capability. The build will be modified to mimic the previous behavior, so that passing feature flags via CFLAGS will pick them up properly.

If you'd like a quicker way to verify this fix than to re-run the whole build and test:

$ ./configure ...your --flags and CFLAGS...
$ make testfixure "OPTIONS=-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT"
$ ./testfixture test/e_delete.test

(It may tell you that there were -1 malloc() calls, but that's a known quirk.)

I have tried out version 3.49.0 but the tests are still failing, as I understood with the patch added recently it shouldn't be necessary to add the custom make command, is that correct?

I've uploaded the build log here: https://bpa.st/HIFQ

I've uploaded the build log here: https://bpa.st/HIFQ

Thank you for that. Something odd is going on with your build.

Just stepping through this to summarize what the logs reveal...

Your CFLAGS are presumably being set via an environment variable instead of being passed to configure:

./configure --prefix=/usr <...huge snip...> --soname=legacy <no CFLAGS>

and yet:

C compiler... x86_64-pc-linux-gnu-gcc -O2 -pipe  -DSQLITE_ENABLE_API_ARMOR <...huge snip...> -DSQLITE_USE_URI

There's nothing inherently wrong with that, but the end of the configure process says:

Library feature flags: -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_THREADSAFE=1

Which is the core of the problem: it has not recognized the feature flags passed in via CFLAGS. That should have emitted the following (taken from my system after doing export CFLAGS=...your...list...of...cflags...):

Library feature flags: -DSQLITE_ENABLE_API_ARMOR <...huge snip...> -DSQLITE_THREADSAFE=1

Your case can be reproduced by not using export for the CFLAGS:

$ export CFLAGS=...
$ ./configure ...
# ^^^ works, but...
$ unset CFLAGs
$ CFLAGS=...
$ ./configure ...
# ^^^ same effect you're reporting

(Pedantic sidebar: if you're restricted to Bourne shell compatibility, the export has to come after the assignment: CFLAGS=...; export CFLAGS)

Based on that, i can offer these workarounds:

1. Add "export" to your CFLAGS or
2. Pass the CFLAGS directly to ./configure ... 'CFLAGS=...'
3. (added later): invoke configure like: CFLAGS='...' ./configure ...flags

Any of those should resolve this for you.

What's not currently clear to me is how the base-most compiler checks are seeing non-exported CFLAGS but the later tests which extract the feature flags from CFLAGS are not, and i'll look for a solution to that mystery. Edit: that was a misdiagnosis on my part. The CFLAGS are not being detected by those tests if they're not exported. Even so, it's still a mystery why the posted configure log clearly shows the CFLAGS being recognized in the cc/c++ tests and not in the later tests.

Thank you for looking into this! You're right — the flags weren't being detected, and I hadn't noticed that earlier. The issue was that the feature flags weren't being passed to configure, we were instead adding them to CPPFLAGS, which is exported to the environment, see Gentoo ebuild.

Obvously this worked before, but perhaps I missed something when adapting the Gentoo package to the build system changes.

I modified the setup to add all the flags to CFLAGS instead of CPPFLAGS, and that successfully resolved the issue.

Should we add everything to CFLAGS rather than CPPFLAGS as was done before?

Should we add everything to CFLAGS rather than CPPFLAGS as was done before?

Excellent question. There was some confusion on that point when porting the build and i left this reminder-to-self in Makefile.in:

# Historical note: the pre-3.48 build only honored CPPFLAGS at
# configure-time, and expanded them into the generated Makefile. There
# are, in that build, no uses of CPPFLAGS in the configure-expanded
# Makefile. Ergo: if a client configures with CPPFLAGS=... and then
# explicitly passes CFLAGS=... to make, the CPPFLAGS will be
# lost. That behavior is retained in 3.48+.
#
CFLAGS = @CFLAGS@ @CPPFLAGS@

And a similar note in main.mk:

# Historical note: the pre-3.48 build does not honor CPPFLAGS passed
# to make, so we do not do so here. Both the legacy and 3.48+ builds
# support CPPFLAGS passed at configure-time, and combines them with
# the configure-time CFLAGS.

In other words: pass CFLAGS and/or CPPFLAGS to configure and they'll be combined into CFLAGS via Makefile.in.