Ctrl-Z too pretty for published Linux 3.40.1 CLI
(1) By Larry Brasfield (larrybr) on 2023-01-04 00:47:14 [link] [source]
I'm posting this in case anybody else has seen the same and knows how or why it happens.
For a few years, I have regularly used various builds of the SQLite CLI on *Nix systems, including images taken from the download page for a few Linux systems. I am fond of using job control to suspend execution of command-line utilities, including the CLI, normally effected with control-Z.
To my great surprise, after putting the recently downloaded version 3.40.1 CLI into ~/bin, and running it, I find that control-Z no longer suspends its execution. Instead, the line reader echoes "�" in place of the usual "^Z" and keeps going, complaining about that "�" if it's not backed off.
The CLI built from the same source, using plain "./configure", acts like all the other builds except this recalcitrant one with respect to suspension (aka "Stopping") via the keyboard. The unstoppable one shows most of the same compile options, excepting this list of deltas (+ for unstoppable, - for normal), none of which should have such an effect:
+COMPILER=gcc-5.2.0
-COMPILER=gcc-11.3.0
+ENABLE_FTS5
+ENABLE_GEOPOLY
+ENABLE_PREUPDATE_HOOK
-HAVE_ISNAN
+MUTEX_OMIT
-MUTEX_PTHREADS
+THREADSAFE=0
-THREADSAFE=1
.
The most significant difference, I suspected, is that the downloaded CLI is a 32-bit executable while the ones which I usually use are locally built as 64-bit executables. But when I build the 32-bit CLI locally1, it is able to be stopped normally via control-Z.
Is anybody else seeing this behavior? Is it something strange about my fairly stock Xubuntu setup? Or is the project-distributed sqlite3 somehow to blame?
- ^ The file utility says: "ELF 32-bit LSB pie executable, Intel 80386 ..."
(2) By Keith Medcalf (kmedcalf) on 2023-01-04 01:04:18 in reply to 1 [link] [source]
On Windows CTRL-Z is end-of-file (CTRL-D on Linux machines).
Is it possible that one of the builds is "cooking" the stdin stream (aka readline or equivalent) while the other is not?
(3) By Larry Brasfield (larrybr) on 2023-01-04 01:22:31 in reply to 2 [link] [source]
I have not tried the 32-bit Linux-targetted CLI on WSL, yet.1 That might be a useful datapoint, although I am unsure what could be made of either result.
I know the CLI's "explicit" treatment of its I/O streams pretty well, which is no treatment in the way of "cooking" or "going raw" (in the stty sense.) But maybe the distributed images have been built with a strange readline equivalent that does some such machination. (That would be a shame because it is normally not necessary to mess with stdin settings to make line editing work for interactive scenarios. Forcing the buffering mode would be the most that a line-collect-edit library should have to do unless it means to do things in 2-D, which would be way beyond its charter IMO.)
Do you see the control-Z resistance on either a genuine Linux system or WSL?
- ^ The annoying behavior I report in post #1 is exhibited by the ELF 32-bit i836 image for Linux taken from the project download page.
(4) By Keith Medcalf (kmedcalf) on 2023-01-04 01:48:11 in reply to 3 [link] [source]
I don't use WSL. The only Linux I have access to (at the moment) is a VM running CentOS Stream. configuring the build with --disable-readline --disable-ncurses or the default (equivalent to --enable-readline --enable-ncurses) job control works (this is x64 -- I do not have x86 libraries installed).
(5) By anonymous on 2023-01-04 04:58:58 in reply to 1 [link] [source]
On Xubuntu: lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.1 LTS Release: 22.04 Codename: jammy Distributed binary for sqlite (sha3: ee3f1e027637734c0c2dbc33d5e8fa5aba6f54cf4130fb8247cf37030ba1bc24): $ date Tue 03 Jan 2023 08:51:12 PM PST $ ./sqlite3 SQLite version 3.40.1 2022-12-28 14:03:47 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> select sqlite_version()�; Parse error: unrecognized token: "�" select sqlite_version()�; ^--- error here sqlite> and From an earlier, pre-release build from source (not amalgamation): $ date Tue 03 Jan 2023 08:53:50 PM PST $ ./sqlite3 SQLite version 3.41.0 2022-12-21 14:13:49 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> select sqlite_version()^Z [1]+ Stopped ./sqlite3 I can do a stock build of the latest source, but from the above quick test, there appears to be a difference between those them on the same Xubuntu system. I'll recompile and post from released source which should rule out your Xubuntu.
(6) By anonymous on 2023-01-04 05:14:16 in reply to 1 [link] [source]
I downloaded the 3.41.1 source .zip then the stock linux build instructions (on same XUbuntu system as previous post): mkdir bld ;# Build will occur in a sibling directory cd bld ;# Change to the build directory ../sqlite/configure ;# Run the configure script make ;# Run the makefile. and..... $ configure (now has a warning which previous builds did not): configure: WARNING: Libtool does not cope well with whitespace in `pwd` and $ make make: *** No rule to make target '/home/peon/Desktop/sqlite/sqlite', needed by 'Makefile'. Stop. I imagine you should be able to replicate the above.
(7) By anonymous on 2023-01-04 05:39:46 in reply to 1 [link] [source]
I just ran through both again. On the same Xubuntu system (as two previous posts), that 3.40.1 pre-release 40549bac builds fine with $ cd bld $ ../sqlite/configure $ make while 3.41.1 does not (with the errors I noted). and the released linux binary has the behavior with ^Z you noted (as per my previous post). On the successful configure/make, HAVE_READLINE=0, and HAVE_EDITLINE=0 (we do not get that far with 3.41.1 release source). Re-confirmed, the locally re-compilted version 40549bac (pre-release 3.41.1) CLI handled ^Z with "stopped". On the same system, also re-confirmed can compile from amalgamation into several custom applications (FWIW). Hope that info helps.
(8) By Larry Brasfield (larrybr) on 2023-01-04 08:17:25 in reply to 5 [link] [source]
The above and your later experiments are quite helpful. I am running the same Xubuntu release, so your results tell me that this issue is unlikely to be that I have somehow got this system configured in an arcanely weird way.1
My rebuild from the source tagged as 3.40.1, with CFLAGS=-m32, does not exhibit the odd handling of ^Z. So I will examine your other results with an eye toward understanding what it takes to get ^Z-choking behavior.
- ^ Or, it's a weird configuration pitfall but one somebody else has fallen into.
(9) By anonymous on 2023-01-04 21:09:17 in reply to 8 [link] [source]
Can confirm SQLite-863c03be restores the x86_64 build on Xubuntu. The ^Z behaviour in the SQLite-863c03be x86_64 build of the CLI is: SQLite version 3.41.0 2023-01-04 18:32:14 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. sqlite> select sqlite_version()^Z [1]+ Stopped ./sqlite3 also noted the '../sqlite/configure' warning in my earlier post is also eliminated.
(10) By anonymous on 2023-01-04 21:32:41 in reply to 6 [link] [source]
I noticed my typed version number of 3.41.1 should read 3.41.0 (cut and pasted from src/VERSION this time!). I repeated this mistake when typing versions in several posts (my apologies for that very confusing error). The sqlite3 program outputs show the correct versions tested and is otherwise correct.
(11.2) By Larry Brasfield (larrybr) on 2023-01-05 04:01:01 edited from 11.1 in reply to 10 [link] [source]
(Updated twice to reflect new data.)
If I understand your results correctly, you have not replicated the not-stopped-by-control-Z behavior with your own builds, but see it with the build taken from the download page.
I have downloaded builds going back to version 3.38.0, all of which evince the same swallowing (and choking upon) control-Z behavior.
My conclusion is that this is due to something special about the particular machine on which those builds are done. I doubt it is the (7+ years) old compiler, but its age makes me suspect that the linked-in line editing library is responsible. I will take this up with the builder.
Thanks for helping get this sorted.
(Added for 11.1:)
I have found that a running, control-Z-swallowing sqlite3 session can be made compliant to stop requests thusly:
stty -F <terminal_device> isig
, where "<terminal_device>" is whatever the tty utility says it is for the terminal running that session.1 I think this shows that the stop-resistant CLI is doctoring tty settings somehow.
(Added for 11.2:)
Reading code in the linenoise "library", I see that this is exactly what is happening. The downloadable CLI must be built with that line editing package. This also explains why control-C puts an immediate end to that CLI's processing rather than (perhaps) interrupting a long query and having to be repeated twice more to end the process.
I believe this resolves the mystery part of this puzzle. I've become spoiled by the more sophisticated readline and equivalents.
- ^ I have yet to figure out how to get this stty call done in the background by something started concurrently with the sqlite3 CLI.
(12) By anonymous on 2023-01-05 05:10:49 in reply to 11.2 [link] [source]
Yes, that is correct regarding what my results showed, local builds(happens to be 64 bit) have no problem and have -DHAVE_READLINE=0 -DHAVE_EDITLINE=0 in the make stdout. With the configure problem fixed, the local build of the latest CLI (also 64 bit) was also fine, and "stopped" on ^Z properly. And yes, the downloaded CLI binary (happens to be 32 bit) had the ^Z stop-resistant behaviour. Your description sounds like it matches what was happening. Again, apologies about the versions mix up (3.40.1 & 3.41.0 is dyslexian nightmare, and I had other obligations which made things a bit rushed). Looking forward to a new version number :)
(13) By Rowan Worth (sqweek) on 2023-01-05 06:31:29 in reply to 11.2 [link] [source]
I also came across linenoise while investigating this, and specifically this bug report:
https://github.com/antirez/linenoise/issues/141
It's reported fixed in Jan 2018, so if the build environment is 7 years old then simply updating linenoise may be sufficient.
(14) By Larry Brasfield (larrybr) on 2023-01-05 09:39:03 in reply to 13 [link] [source]
That's a great tip. My reading of linenoise.c was directly from the author's GitHub page, so it appears that the fix report is more of a "there is a simple way to fix this" report. But it is an easy and obvious fix.
Thanks.
(15) By Rowan Worth (sqweek) on 2023-01-06 04:28:43 in reply to 14 [source]
Oh right my bad! The github issue included the patch and a "merged in commit XXXXXXX" message and I jumped to conclusions, failing to notice that said commit belongs to an entirely different fork/repo and the patch has not been applied to the canonical linenoise source :)