Building SQLite branch-3.47 in Termux on Android fails.

checkout: e3c27c6d8a8610ac4ad8ad95e0958ffad05fd58a 2024-10-24 19:31:17 UTC parent: 31e104cc40e335108b57afe8184782ad389b1e8e 2024-10-24 16:02:25 UTC tags: branch-3.47 comment: Patch sqlite3_rsync.c to disable undesirable automatic line ending conversions on Windows. (user: drh)

../configure --disable-tcl

checking build system type... armv8l-unknown-linux-gnueabi checking host system type... armv8l-unknown-linux-gnueabi checking for gcc... gcc checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ISO C89... none needed checking for a sed that does not truncate output... /data/data/com.termux/files/usr/bin/sed checking for grep that handles long lines and -e... /data/data/com.termux/files/usr/bin/grep checking for egrep... /data/data/com.termux/files/usr/bin/grep -E checking for fgrep... /data/data/com.termux/files/usr/bin/grep -F checking for ld used by gcc... /data/data/com.termux/files/usr/bin/ld checking if the linker (/data/data/com.termux/files/usr/bin/ld) is GNU ld... yes checking for BSD- or MS-compatible name lister (nm)... /data/data/com.termux/files/usr/bin/nm -B checking the name lister (/data/data/com.termux/files/usr/bin/nm -B) interface... BSD nm checking whether ln -s works... yes checking the maximum length of command line arguments... 32768 checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... no checking for /data/data/com.termux/files/usr/bin/ld option to reload object files... -r checking for objdump... objdump checking how to recognize dependent libraries... pass_all checking for ar... ar checking for strip... strip checking for ranlib... ranlib checking command to parse /data/data/com.termux/files/usr/bin/nm -B output from gcc object... ok checking how to run the C preprocessor... gcc -E checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking for dlfcn.h... yes checking for objdir... .libs checking if gcc supports -fno-rtti -fno-exceptions... yes checking for gcc option to produce PIC... -fPIC -DPIC checking if gcc PIC flag -fPIC -DPIC works... yes checking if gcc static flag -static works... no checking if gcc supports -c -o file.o... yes checking if gcc supports -c -o file.o... (cached) yes checking whether the gcc linker (/data/data/com.termux/files/usr/bin/ld) supports shared libraries... yes checking whether -lc should be explicitly linked in... no checking dynamic linker characteristics... GNU/Linux ld.so checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes checking whether to build shared libraries... yes checking whether to build static libraries... yes checking for a BSD-compatible install... /data/data/com.termux/files/usr/bin/install -c checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... 64 checking for int8_t... yes checking for int16_t... yes checking for int32_t... yes checking for int64_t... yes checking for intptr_t... yes checking for uint8_t... yes checking for uint16_t... yes checking for uint32_t... yes checking for uint64_t... yes checking for uintptr_t... yes checking for sys/types.h... (cached) yes checking for stdlib.h... (cached) yes checking for stdint.h... (cached) yes checking for inttypes.h... (cached) yes checking malloc.h usability... yes checking malloc.h presence... yes checking for malloc.h... yes checking for fdatasync... yes checking for gmtime_r... yes checking for isnan... yes checking for localtime_r... yes checking for localtime_s... no checking for malloc_usable_size... yes checking for strchrnul... yes checking for usleep... yes checking for utime... yes checking for pread... yes checking for pread64... yes checking for pwrite... yes checking for pwrite64... yes ../configure: 10337: test: x: unexpected operator unable to run tests because of --disable-tcl ../configure: 10456: test: x: unexpected operator configure: Version set to 3.47 configure: Release set to 3.47.0 checking for WASI SDK directory... no checking whether to support threadsafe operation... yes checking for library containing pthread_create... none required checking for library containing pthread_mutexattr_init... none required checking whether to support shared library linked as release mode or not... no checking whether to use an in-ram database for temporary tables... no checking if executables have the .exe suffix... unknown checking for library containing readline... -ledit not using linenoise checking for library containing fdatasync... none required checking build type... release checking zlib.h usability... yes checking zlib.h presence... yes checking for zlib.h... yes checking for library containing deflate... -lz checking for library containing dlopen... none required checking whether to support math functions... yes checking for library containing ceil... -lm checking whether to support JSON functions... yes checking whether to support MEMSYS5... no checking whether to support MEMSYS3... no checking whether to support FTS3... no checking whether to support FTS4... no checking whether to support FTS5... no checking whether to support LIMIT on UPDATE and DELETE statements... no checking whether to support GEOPOLY... no checking whether to support RTREE... no checking whether to support SESSION... no configure: creating ./config.status config.status: creating Makefile config.status: creating sqlite3.pc config.status: creating sqlite_cfg.h config.status: executing libtool commands

make "CFLAGS= -O2 -DSQLITE_OS_UNIX=1"

gcc -g -O2 -o mksourceid /data/data/com.termux/files/home/project/SQLiteDEV/bld/../tool/mksourceid.c sh /data/data/com.termux/files/home/project/SQLiteDEV/bld/../tool/cktclsh.sh 8.4 /data/data/com.termux/files/home/project/SQLiteDEV/bld/../tool/cktclsh.sh: 5: cktclsh8.4.tcl: not found ERROR: This makefile target requires tclsh 8.4 or later. make: *** [Makefile:807: has_tclsh84] Error 1

apt list

... eltclsh/stable 1.19-2 arm libsqlite-tcl/stable 3.46.1 arm tcl/stable,now 8.6.14 arm [installed] tcllib/stable 2.0 all ...

Building branch-3.46 works. It appears something has changed in the way TCL is identified.

Building SQLite branch-3.47 in Termux on Android fails.

It would be really interesting to know whether the trunk builds in that environment. The build has been completely overhauled since the 3.47 release.

If it fails, and you can give us some instructions for reproducing your setup, that environment will be added to those we test the new build process on. If it doesn't fail, then... great :).

recompile with -fPIC

i believe the problem here is that you are overriding the -fPIC which is in the default CFLAGS (but i'm on a mobile device so cannot readily confirm that). Please try passing the same flags via CPPFLAGS instead of CFLAGS.

Thanks, using make "CPPFLAGS= -O2 -DSQLITE_OS_UNIX=1" works, but drops my flags, using make "CFLAGS= -fPIC -O2 -DSQLITE_OS_UNIX=1" however builds with my flags.

If I remember correctly I once overruled CFLAGS because of an unwanted -g parameter.

Unfortunately now I get a shell that doesn't have or understand READLINE - I think I saw another thread about that issue.

But this doesn't fix the issue with branch-3.47 ...

So staying on 3.46.1 and waiting for either 3.48 or a fix for the branch-3.47 issue.

If you are interested in the setup, it is a tablet running Android 11 and Termux. Using the F-Droid version of Termux instead of the Playstore version is recommended and Termux appears to have some issues with Android 12+.

Installed packages are:

apt/stable,now 2.8.1-1 arm [installed] bash-completion/stable,now 2.14.0 all [installed,automatic] bash/stable,now 5.2.37 arm [installed] binutils-bin/stable,now 2.43.1 arm [installed,automatic] binutils-libs/stable,now 2.43.1 arm [installed,automatic] binutils/stable,now 2.43.1 arm [installed] busybox/stable,now 1.36.1-2 arm [installed] bzip2/stable,now 1.0.8-6 arm [installed] ca-certificates/stable,now 1:2024.09.24 all [installed] caddy/stable,now 2.8.4 arm [installed] clang/stable,now 19.1.2 arm [installed,automatic] command-not-found/stable,now 2.4.0-48 arm [installed] coreutils/stable,now 9.5-3 arm [installed] curl/stable,now 8.10.1-2 arm [installed] dash/stable,now 0.5.12 arm [installed] debianutils/stable,now 5.20 arm [installed] dialog/stable,now 1.3-20240307-0 arm [installed,automatic] diffutils/stable,now 3.10 arm [installed] dnsutils/stable,now 9.16.41 arm [installed] dos2unix/stable,now 7.5.2 arm [installed] dpkg/stable,now 1.22.6-1 arm [installed] ed/stable,now 1.20.2 arm [installed] file/stable,now 5.45 arm [installed] findutils/stable,now 4.10.0 arm [installed] fossil/stable,now 2.24 arm [installed] gawk/stable,now 5.3.0 arm [installed] gdbm/stable,now 1.24 arm [installed,automatic] git/stable,now 2.47.0 arm [installed,automatic] glib/stable,now 2.80.5-2 arm [installed,automatic] gnupg/stable,now 2.4.5-3 arm [installed] gpgv/stable,now 2.4.5-3 arm [installed] grep/stable,now 3.11 arm [installed] gzip/stable,now 1.13 arm [installed] inetutils/stable,now 2.4-2 arm [installed] iperf3/stable,now 3.17.1 arm [installed] jq/stable,now 1.7.1 arm [installed] krb5/stable,now 1.21.3 arm [installed,automatic] ldd/stable,now 0.3 arm [installed] ldns/stable,now 1.8.4 arm [installed,automatic] less/stable,now 661 arm [installed] libandroid-glob/stable,now 0.6-2 arm [installed] libandroid-posix-semaphore/stable,now 0.1-3 arm [installed,automatic] libandroid-selinux/stable,now 14.0.0.11 arm [installed,automatic] libandroid-support/stable,now 29 arm [installed] libandroid-utimes/stable,now 0.4 arm [installed,automatic] libassuan/stable,now 3.0.1-2 arm [installed,automatic] libbz2/stable,now 1.0.8-6 arm [installed] libc++/stable,now 27b arm [installed] libcap-ng/stable,now 2:0.8.5 arm [installed,automatic] libcompiler-rt/stable,now 19.1.2 arm [installed,automatic] libcrypt/stable,now 0.2-5 arm [installed] libcurl/stable,now 8.10.1-2 arm [installed] libdb/stable,now 18.1.40-4 arm [installed,automatic] libedit/stable,now 20240517-3.1-0 arm [installed,automatic] libevent/stable,now 2.1.12-2 arm [installed,automatic] libexpat/stable,now 2.6.3 arm [installed,automatic] libffi/stable,now 3.4.6-1 arm [installed,automatic] libgcrypt/stable,now 1.11.0 arm [installed] libgmp/stable,now 6.3.0-1 arm [installed] libgnutls/stable,now 3.8.5 arm [installed,automatic] libgpg-error/stable,now 1.50 arm [installed] libiconv/stable,now 1.17 arm [installed] libicu/stable,now 75.1 arm [installed,automatic] libidn2/stable,now 2.3.7 arm [installed,automatic] libksba/stable,now 1.6.7-1 arm [installed,automatic] libllvm/stable,now 19.1.2 arm [installed,automatic] liblua54/stable,now 5.4.7 arm [installed,automatic] liblz4/stable,now 1.10.0 arm [installed,automatic] liblzma/stable,now 5.6.3 arm [installed] libmd/stable,now 1.1.0 arm [installed,automatic] libmpfr/stable,now 4.2.1 arm [installed] libnettle/stable,now 3.10 arm [installed,automatic] libnghttp2/stable,now 1.64.0 arm [installed] libnghttp3/stable,now 1.6.0 arm [installed,automatic] libnpth/stable,now 1.6-2 arm [installed,automatic] libresolv-wrapper/stable,now 1.1.7-4 arm [installed,automatic] libsmartcols/stable,now 2.40.2-1 arm [installed,automatic] libsodium/stable,now 1.0.20 arm [installed,automatic] libsqlite/stable,now 3.46.1 arm [installed] libssh2/stable,now 1.11.1 arm [installed,automatic] libunbound/stable,now 1.22.0 arm [installed,automatic] libunistring/stable,now 1.3 arm [installed,automatic] libuuid/stable,now 2.40.2-1 arm [installed,automatic] libuv/stable,now 1.49.2 arm [installed,automatic] libxml2/stable,now 2.13.4-1 arm [installed,automatic] libxslt/stable,now 1.1.42-1 arm [installed,automatic] libzip/stable,now 1.11.1 arm [installed,automatic] libzmq/stable,now 4.3.5 arm [installed] lld/stable,now 19.1.2 arm [installed,automatic] llvm/stable,now 19.1.2 arm [installed,automatic] lua54/stable,now 5.4.7 arm [installed] lzip/stable,now 1.24.1 arm [installed] m4/stable,now 1.4.19-4 arm [installed] make/stable,now 4.4.1 arm [installed,automatic] man/stable,now 1.14.6-1 arm [installed] mc/stable,now 4.8.32 arm [installed] mlocate/stable,now 0.26-5 arm [installed] nano/stable,now 8.2 arm [installed] ncurses-ui-libs/stable,now 6.5.20240831-2 arm [installed,automatic] ncurses-utils/stable,now 6.5.20240831-2 arm [installed] ncurses/stable,now 6.5.20240831-2 arm [installed] ndk-sysroot/stable,now 27b arm [installed,automatic] net-tools/stable,now 2.10.0 arm [installed] oniguruma/stable,now 6.9.9 arm [installed,automatic] openssh-sftp-server/stable,now 9.9p1-4 arm [installed,automatic] openssh/stable,now 9.9p1-4 arm [installed] openssl-tool/stable,now 1:3.3.2 arm [installed] openssl/stable,now 1:3.3.2 arm [installed] pass/stable,now 1.7.4-4 all [installed] patch/stable,now 2.7.6-4 arm [installed] pcre2/stable,now 10.44 arm [installed,automatic] pcre/stable,now 8.45-1 arm [installed] perl/stable,now 5.38.2-3 arm [installed,automatic] php/stable,now 8.3.10-1 arm [installed] pinentry/stable,now 1.3.1-1 arm [installed,automatic] pkg-config/stable,now 0.29.2-2 arm [installed,automatic] procps/stable,now 3.3.17-5 arm [installed] psmisc/stable,now 23.7 arm [installed] python-ensurepip-wheels/stable,now 3.12.7-1 all [installed,automatic] python-pip/stable,now 24.3.1 all [installed] python/stable,now 3.12.7-1 arm [installed] rclone/stable,now 1.68.1 arm [installed] readline/stable,now 8.2.13 arm [installed] resolv-conf/stable,now 1.3 arm [installed,automatic] sed/stable,now 4.9-1 arm [installed] sqlite/stable,now 3.46.1 arm [installed] tar/stable,now 1.35 arm [installed] tcl/stable,now 8.6.14 arm [installed] termux-am-socket/stable,now 1.5.0 arm [installed,automatic] termux-am/stable,now 0.8.0-1 all [installed] termux-auth/stable,now 1.5.0 arm [installed,automatic] termux-exec/stable,now 1:1.0 arm [installed] termux-keyring/stable,now 3.12-1 all [installed] termux-licenses/stable,now 2.0-3 all [installed] termux-tools/stable,now 1.44.2 arm [installed] texinfo/stable,now 7.1.1 arm [installed] tidy/stable,now 5.9.14-next-0 arm [installed,automatic] tree/stable,now 2.1.3 arm [installed,automatic] unrar/stable,now 7.0.9 arm [installed] unzip/stable,now 6.0-9 arm [installed] update-info-dir/stable,now 7.1.1 all [installed,automatic] util-linux/stable,now 2.40.2-1 arm [installed] vim-runtime/stable,now 9.1.0800 all [installed,automatic] vim/stable,now 9.1.0800 arm [installed] wget/stable,now 1.24.5 arm [installed] which/stable,now 2.21 arm [installed] wol/stable,now 0.7.1-3 arm [installed] xxhash/stable,now 0.8.2 arm [installed,automatic] xz-utils/stable,now 5.6.3 arm [installed] yq/stable,now 4.44.3 arm [installed] zip/stable,now 3.0-5 arm [installed] zlib/stable,now 1.3.1 arm [installed] zstd/stable,now 1.5.6-2 arm [installed,automatic]

With this trunk can be build!

If I remember correctly I once overruled CFLAGS because of an unwanted -g parameter.

As of the current trunk, overriding CFLAGS is fine and won't nuke your -fPIC. To the best of my knowledge we currently only use -g by default on the source tree's own in-tree tools (code generators, etc.), not the end user deliverables. That said: my knowledge of the build internals for versions <= 3.47 is only superficial - it's possible that that build system used -g more readily.

Unfortunately now I get a shell that doesn't have or understand READLINE - I think I saw another thread about that issue.

readline/stable,now 8.2.13 arm

i don't know whether that package includes the header files. You may need to install a package named (probably) readline-dev. That's just a guess, though.

Please post the output of:

find /usr -type f -name readline.h

If the output is empty then you definitely need the dev package. If the output is not empty, that info may help us improve the readline detection.

But this doesn't fix the issue with branch-3.47 ...

Nope - 3.47 uses a completely different build process.

tool/cktclsh.sh: 5: cktclsh8.4.tcl: not found

The --disable-tcl flag does not eliminate the tree's requirement for TCL. It eliminates the build of the TCL SQLite extension and the related testing tools. TCL is still required for much of the code generation steps, and --disable-tcl does not eliminate that requirement. 3.48's build system eliminates the dependency on TCL for the code generator tools.

The error you're seeing is a side effect of the configure step not having found tclsh, and your configure output definitely points to syntax error in the configure script in that area which looks like a likely culprit:

../configure: 10337: test: x: unexpected operator
unable to run tests because of --disable-tcl
../configure: 10456: test: x: unexpected operator

i don't have an immediate solution for that, and cannot reproduce it locally, but we will look into it.

../configure: 10337: test: x: unexpected operator

We now have a possible culprit: those two invocations of "test" use the == operator, which is accepted on many shells but might not be accepted on yours. If you're able to, please try replacing all three instances of == on those two lines with = then re-run configure.

The affected lines look like:

if test x"${with_tclsh}" == x -a x"${with_tcl}" == x; then

...

if test x"$TCLSH_CMD" == x; then

There are many other instances of == in that file but they are not in test invocations and must not be changed.

We now have a possible culprit: ...

Got a bit ahead of myself. That fix is now in branch-3.47. It might resolve the follow-up tcl error - i'm not certain - but it will resolve the "suspiciously adjacent" syntax errors your shell reports.

Thanks!

That does indeed solve the issue with branch-3.47.

Thanks!

That works too.

My usage of CPFLAGS was originally for suppression of -g - and that was somewhere in the 3.30's (or earlier?).

Tried locate readline.h and got

/data/data/com.termux/files/usr/include/editline/readline.h /data/data/com.termux/files/usr/include/readline/readline.h

(picked the larger of the two).

Tried ../configure --disable-tcl --with-readline-header=/data/data/com.termux/files/usr/include/readline/readline.h

And yes!

After make "CFLAGS= -O2 -DSQLITE_OS_UNIX=1" a working shell with readline.

Tried ../configure --disable-tcl --with-readline-header=...

Fantastic! Thank you for reporting this.

/data/data/com.termux/files/usr/include/editline/readline.h
/data/data/com.termux/files/usr/include/readline/readline.h

(picked the larger of the two).

In this particular case, either one should actually work for long, boring reasons, but for the sake of completeness...

editline/readline.h is part of libedit, a.k.a. editline, which is a BSD-licensed library functionally similar to readline. editline/readline.h contains libedit's readline-compatible API declarations, making it a drop-in replacement for many uses of libreadline but without the viral licensing conditions.

If you're really curious and want to see that in action, you can probably do:

./configure --with-readline-header=/data/data/com.termux/files/usr/include/editline/readline.h --with-readline-ldflags=-ledit

If that builds (it "should" so long as libedit is installed) then running "ldd sqlite3" will show that it's actually linked to libedit, not libreadline.

More simply:

./configure --editline

"should" find and use libedit, but it would not surprise me if that fails in the termux environment for the same reason its automatic detection of readline.h failed (the header's in a weird place).

After make "CFLAGS= -O2 -DSQLITE_OS_UNIX=1" a working shell with readline.

By the way: is there a particular reason you're setting -DSQLITE_OS_UNIX=1? You shouldn't need to set that. i.e. it should build identically without that flag.

That... is a VERY good question.

It was carried over from my MINT21 build environment; that came from my MINT19 build environment; that originated in a MINT18 VM.

And why?

I think it was needed at that time, maybe just for an integrated extension that misbehaved under Linux.

I did not document it unfortunately.

So... removed it and no differences found.

i believe the problem here is that you are overriding the -fPIC which is in the default CFLAGS

That's definitely the case. The current trunk now separately tracks the "core" CFLAGS (those which should not be overridden by make invocations) and will retain -fPIC when clients pass their own CFLAGS.

If you would, please try again with the latest trunk. Also, if you'll tell me how to set up a similar environment i'll set one up rather than continue to pester you to test these changes :). (Edit: you answered that in parallel as our responses crossed each other.)

Please try passing the same flags via CPPFLAGS instead of CFLAGS.

Correction: that won't work because of how CPPFLAGS is used. For historical compatibility, CPPFLAGS is only honored when running ./configure and it's appended to the CFLAGS (which your usage then overrides, nuking the configure-time CPPFLAGS). "Feature or bug" i cannot rightly say, but it's how CPPFLAGS has always been applied in historical builds.