Thank you for the new release and all the updates!

We’re seeing multiple reports in the Gentoo bug tracker of SQLite 3.47 failing to build when the --enable-tcl argument is used. More details are available at https://bugs.gentoo.org/942918, and I’ll include the relevant build log snippet below.

cat /var/tmp/portage/dev-db/sqlite-3.47.0/work/sqlite-src-3470000-abi_x86_64.amd64/src/tclsqlite.c >>tclsqlite3.c
/usr/bin/tclsh8.6 /var/tmp/portage/dev-db/sqlite-3.47.0/work/sqlite-src-3470000-abi_x86_64.amd64/tool/buildtclext.tcl --cc "gcc" -DSQLITE_SOUNDEX -DSQLITE_USE_URI -DSQLITE_SECURE_DELETE  -O2 -march=native -fomit-frame-pointer -pipe -DSQLITE_OS_UNIX=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_API_ARMOR -DSQLITE_ENABLE_BYTECODE_VTAB -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_DBSTAT_VTAB -DSQLITE_ENABLE_DESERIALIZE -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_HIDDEN_COLUMNS -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_ENABLE_NORMALIZE -DSQLITE_ENABLE_OFFSET_SQL_FUNC -DSQLITE_ENABLE_PREUPDATE_HOOK -DSQLITE_ENABLE_RBU -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_GEOPOLY -DSQLITE_ENABLE_SESSION -DSQLITE_ENABLE_STMT_SCANSTATUS -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT -DSQLITE_ENABLE_ICU  
can't read "@": no such variable
    while executing
"subst $cmd"
    invoked from within
"if {$tcl_platform(platform)=="windows"} {
  # We are only able to install, uninstall, and list on Windows.
  # The build process is handled by the Mak..."
    (file "/var/tmp/portage/dev-db/sqlite-3.47.0/work/sqlite-src-3470000-abi_x86_64.amd64/tool/buildtclext.tcl" line 64)

I'll try to find out what exactly is causing this issue, in the meantime if any additional information is needed please let me know. Thanks!

The log file is nice and all, but how do you get it to fail? What is the command that you are using to start the build that is not working? What are the exact options to ./configure? What make target(s) are being used?

Can you reproduce the problem? Can you try using the latest trunk check-in (tarball available from https://sqlite.org/src/tarball/trunk/sqlite-trunk.tar.gz) and see if that works any better for you?

Apologies, I should have given more details in the initial post. The full log is available at https://942918.bugs.gentoo.org/attachment.cgi?id=908001

This is the configure command with all options passed:

./configure --prefix=/usr --build=x86_64-pc-linux-gnu --host=x86_64-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib --datarootdir=/usr/share --disable-static --docdir=/usr/share/doc/sqlite-3.47.0 --htmldir=/usr/share/doc/sqlite-3.47.0/html --with-sysroot=/ --libdir=/usr/lib64 --enable-load-extension --enable-threadsafe --enable-fts5 --disable-debug --disable-editline --enable-readline --with-readline-inc=-I/usr/include/readline --disable-static --enable-tcl --with-tcl=/usr/lib64

After that the source is compiled with make V=1 -j1 HAVE_TCL=1 TCLLIBDIR=/usr/lib64/sqlite-3.47.0, and to install the package we use make V=1 -j1 DESTDIR=/var/tmp/portage/dev-db/sqlite-3.47.0/image HAVE_TCL=1 TCLLIBDIR=/usr/lib64/sqlite-3.47.0 install, which gives the following output:

/usr/lib/portage/python3.12/ebuild-helpers/xattr/install -c -d /var/tmp/portage/dev-db/sqlite-3.47.0/image/usr/lib64
./libtool --mode=install /usr/lib/portage/python3.12/ebuild-helpers/xattr/install -c libsqlite3.la /var/tmp/portage/dev-db/sqlite-3.47.0/image/usr/lib64
libtool: install: /usr/lib/portage/python3.12/ebuild-helpers/xattr/install -c .libs/libsqlite3.so.0.8.6 /var/tmp/portage/dev-db/sqlite-3.47.0/image/usr/lib64/libsqlite3.so.0.8.6
libtool: install: (cd /var/tmp/portage/dev-db/sqlite-3.47.0/image/usr/lib64 && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so.0 || { rm -f libsqlite3.so.0 && ln -s libsqlite3.so.0.8.6 libsqlite3.so.0; }; })
libtool: install: (cd /var/tmp/portage/dev-db/sqlite-3.47.0/image/usr/lib64 && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so || { rm -f libsqlite3.so && ln -s libsqlite3.so.0.8.6 libsqlite3.so; }; })
libtool: install: /usr/lib/portage/python3.12/ebuild-helpers/xattr/install -c .libs/libsqlite3.lai /var/tmp/portage/dev-db/sqlite-3.47.0/image/usr/lib64/libsqlite3.la
libtool: warning: remember to run 'libtool --finish /usr/lib64'
echo '#ifndef USE_SYSTEM_SQLITE' >tclsqlite3.c
cat sqlite3.c >>tclsqlite3.c
echo '#endif /* USE_SYSTEM_SQLITE */' >>tclsqlite3.c
cat /var/tmp/portage/dev-db/sqlite-3.47.0/work/sqlite-src-3470000-abi_x86_64.amd64/src/tclsqlite.c >>tclsqlite3.c
/usr/bin/tclsh8.6 /var/tmp/portage/dev-db/sqlite-3.47.0/work/sqlite-src-3470000-abi_x86_64.amd64/tool/buildtclext.tcl --cc "gcc" -DSQLITE_SOUNDEX -DSQLITE_USE_URI -DSQLITE_SECURE_DELETE  -O2 -march=native -fomit-frame-pointer -pipe -DSQLITE_OS_UNIX=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_API_ARMOR -DSQLITE_ENABLE_BYTECODE_VTAB -DSQLITE_ENABLE_COLUMN_METADATA -DSQLITE_ENABLE_DBPAGE_VTAB -DSQLITE_ENABLE_DBSTAT_VTAB -DSQLITE_ENABLE_DESERIALIZE -DSQLITE_ENABLE_EXPLAIN_COMMENTS -DSQLITE_ENABLE_FTS3 -DSQLITE_ENABLE_FTS3_PARENTHESIS -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_HIDDEN_COLUMNS -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_ENABLE_NORMALIZE -DSQLITE_ENABLE_OFFSET_SQL_FUNC -DSQLITE_ENABLE_PREUPDATE_HOOK -DSQLITE_ENABLE_RBU -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_GEOPOLY -DSQLITE_ENABLE_SESSION -DSQLITE_ENABLE_STMT_SCANSTATUS -DSQLITE_ENABLE_STMTVTAB -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION -DSQLITE_ENABLE_UNLOCK_NOTIFY -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT -DSQLITE_ENABLE_ICU  
can't read "@": no such variable
    while executing

I reproduced the error on my amd64 system (including my development environment as well as a clean one) and a riscv64 development box I have on hand.

Yes, we have a package which pulls from the latest trunk branch, I will try it out and report back if it helps.

I've just tested this again using the latest trunk check-in, and the error no longer appears. I'll try a bisect to find out at what check-in the problem disappears.

EDIT: This didn't appear in my initial testing because the package build recipe required modifications due to the build system changes, but the issue is still present in the latest check-in, executing tclextension-install target in the installation phase errors out with the same message.

executing tclextension-install target in the installation phase errors out with the same message.

Can you please post the contents of your tclConfig.sh file? That's the apparent source of the errant @ sign.

If you're unsure of where to find that file, the configure process in the trunk version will tell you:

Using tclConfig.sh: /home/stephan/prefix/lib/tclConfig.sh

or the generated Makefile will tell you:

$ grep TCL_CONFIG_SH Makefile
TCL_CONFIG_SH = /home/stephan/prefix/lib/tclConfig.sh

That said: buildtclext.tcl does its own lookup of that file, so can hypothetically return a different path on a system which has multiple TCL versions installed.

I guess the @ sign coming from:

That seems very likely, thank you for tracking that down!

We'll have to consider how to deal with (or not) such cases of non-canonical tclConfig.sh content. For comparison's sake, here's the relevant snippet from a canonical copy:

# Base command to use for combining object files into a shared library:
TCL_SHLIB_LD='${CC} ${CFLAGS} ${LDFLAGS} -shared'

That's the command which the build script is expecting to execute.

@Jakov can you please try applying the following diff to your copy and see if it resolves the problem for you?

Index: tool/buildtclext.tcl
==================================================================
--- tool/buildtclext.tcl
+++ tool/buildtclext.tcl
@@ -140,12 +140,13 @@
   if {[string length $OPTS]>1} {
     append LDFLAGS $OPTS
   }
-  set CMD [subst $cmd]
   if {$TCLMAJOR>8} {
     set OUT libtcl9sqlite$VERSION.$SUFFIX
   } else {
     set OUT libsqlite$VERSION.$SUFFIX
   }
+  set @ $OUT; # workaround for https://sqlite.org/forum/forumpost/6b1af7eb10884373
+  set CMD [subst $cmd]
 }
   
 # Show information about prior installs

It seems building fine here, but it does not recognize DESTDIR when installing:

+ /usr/bin/make install DESTDIR=/builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64 'INSTALL=/usr/bin/install -p'
/usr/bin/install -p -d /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64
./libtool --mode=install /usr/bin/install -p libsqlite3.la /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64
libtool: install: /usr/bin/install -p .libs/libsqlite3.so.0.8.6 /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64/libsqlite3.so.0.8.6
libtool: install: (cd /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64 && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so.0 || { rm -f libsqlite3.so.0 && ln -s libsqlite3.so.0.8.6 libsqlite3.so.0; }; })
libtool: install: (cd /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64 && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so || { rm -f libsqlite3.so && ln -s libsqlite3.so.0.8.6 libsqlite3.so; }; })
libtool: install: /usr/bin/install -p .libs/libsqlite3.lai /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64/libsqlite3.la
libtool: install: warning: remember to run `libtool --finish /usr/lib64'
echo '#ifndef USE_SYSTEM_SQLITE' >tclsqlite3.c
cat sqlite3.c >>tclsqlite3.c
echo '#endif /* USE_SYSTEM_SQLITE */' >>tclsqlite3.c
cat /builddir/build/BUILD/sqlite-src-3470000/src/tclsqlite.c >>tclsqlite3.c
tclsh8.6 /builddir/build/BUILD/sqlite-src-3470000/tool/buildtclext.tcl --cc "gcc" -O2 -flto=auto -ffat-lto-objects -g -grecord-gcc-switches -pipe -fstack-protector-strong -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/generic-hardened-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/generic-hardened-ld -DSQLITE_DISABLE_DIRSYNC=1 -DSQLITE_SECURE_DELETE=1 -Wall -fno-strict-aliasing -DSQLITE_OS_UNIX=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_ENABLE_FTS3=3 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_UNLOCK_NOTIFY=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS=1 -DSQLITE_ENABLE_JSON1=1  
None of the directories on $auto_path are writable by this process,
so the installation cannot take place.  Consider running using sudo
to work around this problem.

These are the (unwritable) $auto_path directories:

  *  /usr/share/tcl8.6
  *  /usr/lib64/tcl8.6 /usr/share/tcl8.6 /usr/lib64/tk8.6 /usr/share/tk8.6

It seems building fine here, but [the tcl extension] does not recognize DESTDIR when installing:

Thank you for the report. That's now fixed in the trunk and the 3.47 branch.

Thanks. but it still failed with patched 3.47:

+ mkdir -p /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64//usr/lib64/tcl8.6/sqlite3
+ /usr/bin/make install DESTDIR=/builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64 'INSTALL=/usr/bin/install -p'
/usr/bin/install -p -d /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64
./libtool --mode=install /usr/bin/install -p libsqlite3.la /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64
libtool: install: /usr/bin/install -p .libs/libsqlite3.so.0.8.6 /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64/libsqlite3.so.0.8.6
libtool: install: (cd /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64 && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so.0 || { rm -f libsqlite3.so.0 && ln -s libsqlite3.so.0.8.6 libsqlite3.so.0; }; })
libtool: install: (cd /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64 && { ln -s -f libsqlite3.so.0.8.6 libsqlite3.so || { rm -f libsqlite3.so && ln -s libsqlite3.so.0.8.6 libsqlite3.so; }; })
libtool: install: /usr/bin/install -p .libs/libsqlite3.lai /builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64/usr/lib64/libsqlite3.la
libtool: install: warning: remember to run `libtool --finish /usr/lib64'
echo '#ifndef USE_SYSTEM_SQLITE' >tclsqlite3.c
cat sqlite3.c >>tclsqlite3.c
echo '#endif /* USE_SYSTEM_SQLITE */' >>tclsqlite3.c
cat /builddir/build/BUILD/sqlite-src-3470000/src/tclsqlite.c >>tclsqlite3.c
tclsh8.6 /builddir/build/BUILD/sqlite-src-3470000/tool/buildtclext.tcl --destdir "/builddir/build/BUILDROOT/sqlite-3.47.0-1.x86_64" --cc "gcc" -O2 -flto=auto -ffat-lto-objects -g -grecord-gcc-switches -pipe -fstack-protector-strong -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/generic-hardened-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/generic-hardened-ld -DSQLITE_DISABLE_DIRSYNC=1 -DSQLITE_SECURE_DELETE=1 -Wall -fno-strict-aliasing -DSQLITE_OS_UNIX=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_COLUMN_METADATA=1 -DSQLITE_ENABLE_FTS3=3 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_UNLOCK_NOTIFY=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 -DSQLITE_ENABLE_FTS3_PARENTHESIS=1 -DSQLITE_ENABLE_JSON1=1  
None of the directories on $auto_path are writable by this process,
so the installation cannot take place.  Consider running using sudo
to work around this problem.

These are the (unwritable) $auto_path directories:

  *  /usr/share/tcl8.6
  *  /usr/lib64/tcl8.6 /usr/share/tcl8.6 /usr/lib64/tk8.6 /usr/share/tk8.6
make: *** [Makefile:1603: tclextension-install] Error 1

I guess there is still something wrong with it. Even after I create a directory manually.

Thanks. but it still failed with patched 3.47:

Thank you for the report. Please try again with the tip of the 3.47 branch, which contains two changes:

• Virtual filesystems (namely //zipfs:) are filtered out of the installation target checks.

• The "is-writable" check is now applied differently if DESTDIR is set: if DESTDIR is not writable, the installation will fail but if DESTDIR is writable then the installation will assume that the target path under DESTDIR is writable and will attempt to install there (failing at a later stage if the downstream mkdir fails).

Thank you, the build is fixed now, I have tested and applied the patches to our package

maybe a 3.47.1 release for distro packagers.

Well, with latest 3.47 branch, it installed into /usr/share/tcl8.6/sqlite3.47.0, where I think /usr/lib64/tcl8.6/sqlite3.47.0 should be used, because it contains libsqlite3.47.0.so which is arch-dependent file.

installed into /usr/share/tcl8.6/sqlite3.47.0, where I think /usr/lib64/tcl8.6/sqlite3.47.0 should be used

Which dir is used is decided by the tclsh installation. We simply query it where the library should go, then put it in the first option it gives us. The installer knows nothing about whether any given dir is "arch-dependent" or not.

I don't know what distro or exact build steps you are using, but here on Gentoo the directory is /usr/lib64/tcl8.6/sqlite3.47.0, same as before applied patches