Building SQLite branch-3.47 in Termux on Android fails.

It would be really interesting to know whether the trunk builds in that environment. The build has been completely overhauled since the 3.47 release.

If it fails, and you can give us some instructions for reproducing your setup, that environment will be added to those we test the new build process on. If it doesn't fail, then... great :).

Building SQLite trunk in Termux on Android also fails.

checkout: 1d24a29c6ef05185950ba5c45f2a60a92f12a8e5 2024-10-29 08:42:06 UTC parent: 80ac10ed8d0ee9eaf83a6c43608098c631f5f953 2024-10-29 08:18:45 UTC tags: trunk comment: When either --readline or --editline are explicitly provided but the corresponding feature is not found, fail fatally. If not explicitly requested, simply disable that feature if the lib is not found. (user: stephan)

../configure --disable-tcl

Host System...armv8l-unknown-linux-androideabi Build System...armv8l-unknown-linux-androideabi C compiler... cc C++ compiler... c++ Build C compiler...cc Checking for stdlib.h...ok srcdir = /data/data/com.termux/files/home/project/SQLiteDEV top_srcdir = /data/data/com.termux/files/home/project/SQLiteDEV Configuring SQLite version 3.48.0 This appears to be an out-of-tree build. Looking for install ... /data/data/com.termux/files/usr/bin/install Checking for sys/types.h...ok Checking if -D_FILE_OFFSET_BITS=64 is needed...yes Checking for int8_t...ok Checking for int16_t...ok Checking for int32_t...ok Checking for int64_t...ok Checking for intptr_t...ok Checking for uint8_t...ok Checking for uint16_t...ok Checking for uint32_t...ok Checking for uint64_t...ok Checking for uintptr_t...ok Checking for gmtime_r...ok Checking for isnan...ok Checking for localtime_r...ok Checking for localtime_s...not found Checking for malloc_usable_size...ok Checking for strchrnul...ok Checking for usleep...ok Checking for utime...ok Checking for pread...ok Checking for pread64...ok Checking for pwrite...ok Checking for pwrite64...ok Checking libs for fdatasync...none needed Checking for sys/types.h...(cached) ok Checking for sys/stat.h...ok Checking for dlfcn.h...ok Checking for unistd.h...ok Checking for stdlib.h...(cached) ok Checking for malloc.h...ok Checking for memory.h...ok Checking for string.h...ok Checking for strings.h...ok Checking for inttypes.h...ok Checking for zlib.h...ok Checking libs for deflate...-lz Checking whether the C compiler accepts -rpath /usr/local/lib...yes Build shared library? yes Build static library? yes Use amalgamation for builds? yes Use gcov? no test-runner flags: no Use #line macros in the amalgamation: no SQLITE_DEBUG build? no TCL disabled via --disable-tcl Checking for TCL to use for code generation... Checking for dirent.h...ok Checking for sys/time.h...ok Checking for realpath...ok TCL for code generation: jimsh Support threadsafe operation? yes Checking libs for pthread_create...none needed Checking libs for pthread_mutexattr_init...none needed Use an in-ram database for temporary tables? no Checking for line-editing capability... Line-editing support for the sqlite3 shell: none Checking libs for dlopen...none needed Checking libs for ceil...-lm Enabling math SQL functions -lm Emscripten SDK? not found ICU support is disabled. Feature flags... . − fts4 . − fts5 . − geopoly . − rtree . − session . − update-limit . − memsys5 . − memsys3 . + json Library feature flags: -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_THREADSAFE=1 Shell options: -DSQLITE_HAVE_ZLIB=1 Created Makefile from Makefile.in Created sqlite3.pc from sqlite3.pc.in Created sqlite_cfg.h Source tree is configured! Run make to build it.

make "CFLAGS= -O2 -DSQLITE_OS_UNIX=1"

cc -O2 -DSQLITE_OS_UNIX=1 -DSQLITE_ENABLE_MATH_FUNCTIONS -DSQLITE_THREADSAFE=1 -DNDEBUG -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite -I/usr/local/include -I. -I/data/data/com.termux/files/home/project/SQLiteDEV/src -I/data/data/com.termux/files/home/project/SQLiteDEV/ext/rtree -I/data/data/com.termux/files/home/project/SQLiteDEV/ext/icu -I/data/data/com.termux/files/home/project/SQLiteDEV/ext/fts3 -I/data/data/com.termux/files/home/project/SQLiteDEV/ext/session -I/data/data/com.termux/files/home/project/SQLiteDEV/ext/misc -shared -o libsqlite3.so sqlite3.o -rpath /usr/local/lib -lm -lz ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_version'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(sqlite3_libversion) ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_data_directory'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(sqlite3_shutdown) ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_temp_directory'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(sqlite3_shutdown) ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_temp_directory'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(unixGetTempname) ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_temp_directory'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(sqlite3Pragma) ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_temp_directory'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(sqlite3Pragma) ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_temp_directory'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(sqlite3Pragma) ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_temp_directory'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(sqlite3Pragma) ld.lld: error: relocation R_ARM_REL32 cannot be used against symbol 'sqlite3_version'; recompile with -fPIC > > > defined in sqlite3.o > > > referenced by sqlite3.c > > > sqlite3.o:(versionFunc) cc: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [/data/data/com.termux/files/home/project/SQLiteDEV/main.mk:1335: libsqlite3.so] Error 1

So... different errors, still not going as expected.

recompile with -fPIC

i believe the problem here is that you are overriding the -fPIC which is in the default CFLAGS (but i'm on a mobile device so cannot readily confirm that). Please try passing the same flags via CPPFLAGS instead of CFLAGS.

Thanks, using make "CPPFLAGS= -O2 -DSQLITE_OS_UNIX=1" works, but drops my flags, using make "CFLAGS= -fPIC -O2 -DSQLITE_OS_UNIX=1" however builds with my flags.

If I remember correctly I once overruled CFLAGS because of an unwanted -g parameter.

Unfortunately now I get a shell that doesn't have or understand READLINE - I think I saw another thread about that issue.

But this doesn't fix the issue with branch-3.47 ...

So staying on 3.46.1 and waiting for either 3.48 or a fix for the branch-3.47 issue.

If you are interested in the setup, it is a tablet running Android 11 and Termux. Using the F-Droid version of Termux instead of the Playstore version is recommended and Termux appears to have some issues with Android 12+.

Installed packages are:

apt/stable,now 2.8.1-1 arm [installed] bash-completion/stable,now 2.14.0 all [installed,automatic] bash/stable,now 5.2.37 arm [installed] binutils-bin/stable,now 2.43.1 arm [installed,automatic] binutils-libs/stable,now 2.43.1 arm [installed,automatic] binutils/stable,now 2.43.1 arm [installed] busybox/stable,now 1.36.1-2 arm [installed] bzip2/stable,now 1.0.8-6 arm [installed] ca-certificates/stable,now 1:2024.09.24 all [installed] caddy/stable,now 2.8.4 arm [installed] clang/stable,now 19.1.2 arm [installed,automatic] command-not-found/stable,now 2.4.0-48 arm [installed] coreutils/stable,now 9.5-3 arm [installed] curl/stable,now 8.10.1-2 arm [installed] dash/stable,now 0.5.12 arm [installed] debianutils/stable,now 5.20 arm [installed] dialog/stable,now 1.3-20240307-0 arm [installed,automatic] diffutils/stable,now 3.10 arm [installed] dnsutils/stable,now 9.16.41 arm [installed] dos2unix/stable,now 7.5.2 arm [installed] dpkg/stable,now 1.22.6-1 arm [installed] ed/stable,now 1.20.2 arm [installed] file/stable,now 5.45 arm [installed] findutils/stable,now 4.10.0 arm [installed] fossil/stable,now 2.24 arm [installed] gawk/stable,now 5.3.0 arm [installed] gdbm/stable,now 1.24 arm [installed,automatic] git/stable,now 2.47.0 arm [installed,automatic] glib/stable,now 2.80.5-2 arm [installed,automatic] gnupg/stable,now 2.4.5-3 arm [installed] gpgv/stable,now 2.4.5-3 arm [installed] grep/stable,now 3.11 arm [installed] gzip/stable,now 1.13 arm [installed] inetutils/stable,now 2.4-2 arm [installed] iperf3/stable,now 3.17.1 arm [installed] jq/stable,now 1.7.1 arm [installed] krb5/stable,now 1.21.3 arm [installed,automatic] ldd/stable,now 0.3 arm [installed] ldns/stable,now 1.8.4 arm [installed,automatic] less/stable,now 661 arm [installed] libandroid-glob/stable,now 0.6-2 arm [installed] libandroid-posix-semaphore/stable,now 0.1-3 arm [installed,automatic] libandroid-selinux/stable,now 14.0.0.11 arm [installed,automatic] libandroid-support/stable,now 29 arm [installed] libandroid-utimes/stable,now 0.4 arm [installed,automatic] libassuan/stable,now 3.0.1-2 arm [installed,automatic] libbz2/stable,now 1.0.8-6 arm [installed] libc++/stable,now 27b arm [installed] libcap-ng/stable,now 2:0.8.5 arm [installed,automatic] libcompiler-rt/stable,now 19.1.2 arm [installed,automatic] libcrypt/stable,now 0.2-5 arm [installed] libcurl/stable,now 8.10.1-2 arm [installed] libdb/stable,now 18.1.40-4 arm [installed,automatic] libedit/stable,now 20240517-3.1-0 arm [installed,automatic] libevent/stable,now 2.1.12-2 arm [installed,automatic] libexpat/stable,now 2.6.3 arm [installed,automatic] libffi/stable,now 3.4.6-1 arm [installed,automatic] libgcrypt/stable,now 1.11.0 arm [installed] libgmp/stable,now 6.3.0-1 arm [installed] libgnutls/stable,now 3.8.5 arm [installed,automatic] libgpg-error/stable,now 1.50 arm [installed] libiconv/stable,now 1.17 arm [installed] libicu/stable,now 75.1 arm [installed,automatic] libidn2/stable,now 2.3.7 arm [installed,automatic] libksba/stable,now 1.6.7-1 arm [installed,automatic] libllvm/stable,now 19.1.2 arm [installed,automatic] liblua54/stable,now 5.4.7 arm [installed,automatic] liblz4/stable,now 1.10.0 arm [installed,automatic] liblzma/stable,now 5.6.3 arm [installed] libmd/stable,now 1.1.0 arm [installed,automatic] libmpfr/stable,now 4.2.1 arm [installed] libnettle/stable,now 3.10 arm [installed,automatic] libnghttp2/stable,now 1.64.0 arm [installed] libnghttp3/stable,now 1.6.0 arm [installed,automatic] libnpth/stable,now 1.6-2 arm [installed,automatic] libresolv-wrapper/stable,now 1.1.7-4 arm [installed,automatic] libsmartcols/stable,now 2.40.2-1 arm [installed,automatic] libsodium/stable,now 1.0.20 arm [installed,automatic] libsqlite/stable,now 3.46.1 arm [installed] libssh2/stable,now 1.11.1 arm [installed,automatic] libunbound/stable,now 1.22.0 arm [installed,automatic] libunistring/stable,now 1.3 arm [installed,automatic] libuuid/stable,now 2.40.2-1 arm [installed,automatic] libuv/stable,now 1.49.2 arm [installed,automatic] libxml2/stable,now 2.13.4-1 arm [installed,automatic] libxslt/stable,now 1.1.42-1 arm [installed,automatic] libzip/stable,now 1.11.1 arm [installed,automatic] libzmq/stable,now 4.3.5 arm [installed] lld/stable,now 19.1.2 arm [installed,automatic] llvm/stable,now 19.1.2 arm [installed,automatic] lua54/stable,now 5.4.7 arm [installed] lzip/stable,now 1.24.1 arm [installed] m4/stable,now 1.4.19-4 arm [installed] make/stable,now 4.4.1 arm [installed,automatic] man/stable,now 1.14.6-1 arm [installed] mc/stable,now 4.8.32 arm [installed] mlocate/stable,now 0.26-5 arm [installed] nano/stable,now 8.2 arm [installed] ncurses-ui-libs/stable,now 6.5.20240831-2 arm [installed,automatic] ncurses-utils/stable,now 6.5.20240831-2 arm [installed] ncurses/stable,now 6.5.20240831-2 arm [installed] ndk-sysroot/stable,now 27b arm [installed,automatic] net-tools/stable,now 2.10.0 arm [installed] oniguruma/stable,now 6.9.9 arm [installed,automatic] openssh-sftp-server/stable,now 9.9p1-4 arm [installed,automatic] openssh/stable,now 9.9p1-4 arm [installed] openssl-tool/stable,now 1:3.3.2 arm [installed] openssl/stable,now 1:3.3.2 arm [installed] pass/stable,now 1.7.4-4 all [installed] patch/stable,now 2.7.6-4 arm [installed] pcre2/stable,now 10.44 arm [installed,automatic] pcre/stable,now 8.45-1 arm [installed] perl/stable,now 5.38.2-3 arm [installed,automatic] php/stable,now 8.3.10-1 arm [installed] pinentry/stable,now 1.3.1-1 arm [installed,automatic] pkg-config/stable,now 0.29.2-2 arm [installed,automatic] procps/stable,now 3.3.17-5 arm [installed] psmisc/stable,now 23.7 arm [installed] python-ensurepip-wheels/stable,now 3.12.7-1 all [installed,automatic] python-pip/stable,now 24.3.1 all [installed] python/stable,now 3.12.7-1 arm [installed] rclone/stable,now 1.68.1 arm [installed] readline/stable,now 8.2.13 arm [installed] resolv-conf/stable,now 1.3 arm [installed,automatic] sed/stable,now 4.9-1 arm [installed] sqlite/stable,now 3.46.1 arm [installed] tar/stable,now 1.35 arm [installed] tcl/stable,now 8.6.14 arm [installed] termux-am-socket/stable,now 1.5.0 arm [installed,automatic] termux-am/stable,now 0.8.0-1 all [installed] termux-auth/stable,now 1.5.0 arm [installed,automatic] termux-exec/stable,now 1:1.0 arm [installed] termux-keyring/stable,now 3.12-1 all [installed] termux-licenses/stable,now 2.0-3 all [installed] termux-tools/stable,now 1.44.2 arm [installed] texinfo/stable,now 7.1.1 arm [installed] tidy/stable,now 5.9.14-next-0 arm [installed,automatic] tree/stable,now 2.1.3 arm [installed,automatic] unrar/stable,now 7.0.9 arm [installed] unzip/stable,now 6.0-9 arm [installed] update-info-dir/stable,now 7.1.1 all [installed,automatic] util-linux/stable,now 2.40.2-1 arm [installed] vim-runtime/stable,now 9.1.0800 all [installed,automatic] vim/stable,now 9.1.0800 arm [installed] wget/stable,now 1.24.5 arm [installed] which/stable,now 2.21 arm [installed] wol/stable,now 0.7.1-3 arm [installed] xxhash/stable,now 0.8.2 arm [installed,automatic] xz-utils/stable,now 5.6.3 arm [installed] yq/stable,now 4.44.3 arm [installed] zip/stable,now 3.0-5 arm [installed] zlib/stable,now 1.3.1 arm [installed] zstd/stable,now 1.5.6-2 arm [installed,automatic]

With this trunk can be build!

If I remember correctly I once overruled CFLAGS because of an unwanted -g parameter.

As of the current trunk, overriding CFLAGS is fine and won't nuke your -fPIC. To the best of my knowledge we currently only use -g by default on the source tree's own in-tree tools (code generators, etc.), not the end user deliverables. That said: my knowledge of the build internals for versions <= 3.47 is only superficial - it's possible that that build system used -g more readily.

Unfortunately now I get a shell that doesn't have or understand READLINE - I think I saw another thread about that issue.

readline/stable,now 8.2.13 arm

i don't know whether that package includes the header files. You may need to install a package named (probably) readline-dev. That's just a guess, though.

Please post the output of:

find /usr -type f -name readline.h

If the output is empty then you definitely need the dev package. If the output is not empty, that info may help us improve the readline detection.

But this doesn't fix the issue with branch-3.47 ...

Nope - 3.47 uses a completely different build process.

tool/cktclsh.sh: 5: cktclsh8.4.tcl: not found

The --disable-tcl flag does not eliminate the tree's requirement for TCL. It eliminates the build of the TCL SQLite extension and the related testing tools. TCL is still required for much of the code generation steps, and --disable-tcl does not eliminate that requirement. 3.48's build system eliminates the dependency on TCL for the code generator tools.

The error you're seeing is a side effect of the configure step not having found tclsh, and your configure output definitely points to syntax error in the configure script in that area which looks like a likely culprit:

../configure: 10337: test: x: unexpected operator
unable to run tests because of --disable-tcl
../configure: 10456: test: x: unexpected operator

i don't have an immediate solution for that, and cannot reproduce it locally, but we will look into it.

../configure: 10337: test: x: unexpected operator

We now have a possible culprit: those two invocations of "test" use the == operator, which is accepted on many shells but might not be accepted on yours. If you're able to, please try replacing all three instances of == on those two lines with = then re-run configure.

The affected lines look like:

if test x"${with_tclsh}" == x -a x"${with_tcl}" == x; then

...

if test x"$TCLSH_CMD" == x; then

There are many other instances of == in that file but they are not in test invocations and must not be changed.

We now have a possible culprit: ...

Got a bit ahead of myself. That fix is now in branch-3.47. It might resolve the follow-up tcl error - i'm not certain - but it will resolve the "suspiciously adjacent" syntax errors your shell reports.

Thanks!

That does indeed solve the issue with branch-3.47.

Thanks!

That works too.

My usage of CPFLAGS was originally for suppression of -g - and that was somewhere in the 3.30's (or earlier?).

Tried locate readline.h and got

/data/data/com.termux/files/usr/include/editline/readline.h /data/data/com.termux/files/usr/include/readline/readline.h

(picked the larger of the two).

Tried ../configure --disable-tcl --with-readline-header=/data/data/com.termux/files/usr/include/readline/readline.h

And yes!

After make "CFLAGS= -O2 -DSQLITE_OS_UNIX=1" a working shell with readline.

Tried ../configure --disable-tcl --with-readline-header=...

Fantastic! Thank you for reporting this.

/data/data/com.termux/files/usr/include/editline/readline.h
/data/data/com.termux/files/usr/include/readline/readline.h

(picked the larger of the two).

In this particular case, either one should actually work for long, boring reasons, but for the sake of completeness...

editline/readline.h is part of libedit, a.k.a. editline, which is a BSD-licensed library functionally similar to readline. editline/readline.h contains libedit's readline-compatible API declarations, making it a drop-in replacement for many uses of libreadline but without the viral licensing conditions.

If you're really curious and want to see that in action, you can probably do:

./configure --with-readline-header=/data/data/com.termux/files/usr/include/editline/readline.h --with-readline-ldflags=-ledit

If that builds (it "should" so long as libedit is installed) then running "ldd sqlite3" will show that it's actually linked to libedit, not libreadline.

More simply:

./configure --editline

"should" find and use libedit, but it would not surprise me if that fails in the termux environment for the same reason its automatic detection of readline.h failed (the header's in a weird place).

After make "CFLAGS= -O2 -DSQLITE_OS_UNIX=1" a working shell with readline.

By the way: is there a particular reason you're setting -DSQLITE_OS_UNIX=1? You shouldn't need to set that. i.e. it should build identically without that flag.

That... is a VERY good question.

It was carried over from my MINT21 build environment; that came from my MINT19 build environment; that originated in a MINT18 VM.

And why?

I think it was needed at that time, maybe just for an integrated extension that misbehaved under Linux.

I did not document it unfortunately.

So... removed it and no differences found.

i believe the problem here is that you are overriding the -fPIC which is in the default CFLAGS

That's definitely the case. The current trunk now separately tracks the "core" CFLAGS (those which should not be overridden by make invocations) and will retain -fPIC when clients pass their own CFLAGS.

If you would, please try again with the latest trunk. Also, if you'll tell me how to set up a similar environment i'll set one up rather than continue to pester you to test these changes :). (Edit: you answered that in parallel as our responses crossed each other.)

Please try passing the same flags via CPPFLAGS instead of CFLAGS.

Correction: that won't work because of how CPPFLAGS is used. For historical compatibility, CPPFLAGS is only honored when running ./configure and it's appended to the CFLAGS (which your usage then overrides, nuking the configure-time CPPFLAGS). "Feature or bug" i cannot rightly say, but it's how CPPFLAGS has always been applied in historical builds.