SQLite Forum

sqlite3 history error in vscode terminal
Login

sqlite3 history error in vscode terminal

(1) By curmudgeon on 2021-02-18 11:52:30 [link] [source]

I'm Using the vscode wsl terminal with distro Ubuntu 20.04.

If I install sqlite3 with 'sudo apt install sqlite3' I get version 3.33 and, when running sqlite3, the arrow up/down keys allow me to scroll through the history of previous commands and the left/right keys allow me to move through the current command to correct errors. 

If I use the sqlite3 executable created by the configure and make scripts in the trunk I get version 3.35 but pressing arrow up, arrow down, arrow left, arrow right while running sqlite3 shows

sqlite> ^[[A^[[B^[[D^[[C

There is another problem in vscode terminal with both the aforementioned versions in that if you're on the bottom line and enter a particularly long command the text wraps and then overwrites the bottom line rather than scroll the window up and continue the text on the new bottom line. This I suspect is a vscode problem and I've reported it on github but I'm not sure about the arrow key problem.

(2) By Gunter Hick (gunter_hick) on 2021-02-18 12:22:50 in reply to 1 [link] [source]

Looks like a configuration error in the scripts that is omitting the readline module for editing the recall buffer or causing a wrong version to be used.

(3) By curmudgeon on 2021-02-18 18:45:35 in reply to 2 [link] [source]

I wouldn't have a clue how to check if that was the case Gunter.

Could anyone using 3.35 directly on linux let me know if it's displaying any of these traits in the bash shell.

(4) By Larry Brasfield (larrybr) on 2021-02-18 19:53:40 in reply to 3 [link] [source]

(On WSL, which is Ubuntu 20.04 :) I see arrows acting the same way from v3.34.0 , started from bash. I don't think bash has anything to say about that. Bash itself handles arrows fine.

(6) By curmudgeon on 2021-02-19 07:55:46 in reply to 4 [link] [source]

Sorry Larry, I'm unsure if you're saying the arrow keys work as expected in 3.34 or they're printing ^[[A^[[B^[[D^[[C as on mine.

(10) By Larry Brasfield (larrybr) on 2021-02-19 11:46:50 in reply to 6 [link] [source]

I meant "same as presented problem", ie. arrow keys echoed as that rendered escape sequence rather than interpreted, swallowed and acted upon.

(5) By Warren Young (wyoung) on 2021-02-19 00:10:57 in reply to 2 [link] [source]

You don't have to guess; the configure script tells you:

checking for readline in -lreadline... no
checking for readline.h usability... no

...and so on. It's telling you it didn't find the GNU Readline library headers and library, so you don't get line editing support. This is called out at the end of step 2 in the compilation instructions.

The SQLite configure script can also look for BSD editline, sometimes called libedit, which has a similar API under a more liberal license.

On Ubuntu, the solution is sudo apt install libreadline-dev.

(7) By curmudgeon on 2021-02-19 09:44:52 in reply to 5 [link] [source]

Thanks Warren. I was sending the output to /dev/null thinking everything was fine. I installed libreadline-dev. I still get these messages

checking for library containing readline... no
checking for library containing tgetent... -lncurses
checking for readline in -lreadline... yes
checking readline.h usability... no
checking readline.h presence... no
checking for readline.h... no
checking for /usr/include/readline.h... no
checking for /usr/include/readline/readline.h... yes

Is there something else I'm missing? Either way, the arrow keys do now scroll through the history. I'm assuming When I installed 3.33 using 'sudo apt install sqlite3' ubuntu automatically installed libreadline-dev and then uninstalled it when I purged as otherwise I'd be wondering why 3.33 scrolled through the history without me installing libreadline-dev.

Thanks Warren.

PS It did nothing for bottom line error which I assume is a vscode / wsl problem. There's a video of it at
https://github.com/microsoft/vscode/issues/116174

(8) By Warren Young (wyoung) on 2021-02-19 10:23:41 in reply to 7 [link] [source]

I still get these messages

One "yes" for the library, one "yes" for the header. What's the problem?

You were maybe expecting it to find the library and the headers in each and every place it checked, redundantly? "No" is only a problem when it's "no" for every check. When you're looking for your keys, you only have to find them in one place, not everywhere you look.

the arrow keys do now scroll through the history

So why are you borrowing trouble?

I'm assuming When I installed 3.33 using 'sudo apt install sqlite3' ubuntu automatically installed libreadline-dev

You don't need me to tell you that's not the case. You already had the evidence saying it doesn't.

Where your assumptions went unfounded is that on Ubuntu you only need the libreadline package to run a pre-built binary built against libreadline the library. You need libreadline-dev to build such binaries. That package depends on the libreadline package, so by installing the -dev package, you get the library package, too.

But that's Ubuntu. Other OS packaging systems use different rules. Homebrew bundles the library and development files in a single package, for instance.

It did nothing for bottom line error

"Microsoft software fails to implement ANSI X3.64 correctly. Film at 11."

(9) By curmudgeon on 2021-02-19 11:37:20 in reply to 8 [link] [source]

Thanks Warren, I now realise why 3.33 worked. Despite googling I'm none the wiser about your final sentence but presume it means it's ms's fault and I'll just have to live with it while using linux through wsl.

(11.3) By Warren Young (wyoung) on 2021-02-19 17:35:45 edited from 11.2 in reply to 9 [source]

I'm none the wiser about your final sentence

Microsoft has a long history of providing ANSI X3.64 terminal support — confusingly abbreviated "ANSI", as if that body only ever did one standard — that's slow, incomplete, missing, and/or late compared to its contemporary implementations, so that it's almost always better to use a third-party or hand-rolled solution rather than Microsoft's latest offering.

Most recently, we see this in the assorted brokennesses of WSL1's terminal handling, which they're only just now beginning to correct with the combination of WSL2 and Windows Terminal. As welcome an improvement as that last is over the stupendously creaky Windows Console, it's still buggy and incomplete with respect to the modern ANSI X3.64 descendants on Linux, BSD, and macOS.

I bring up the other OSes not to tell you that you must switch to one of them to get this right but to point out that ANSI X3.64 has been built upon again and again by both de jure and de facto standards since the mid-1970s. Microsoft didn't even try to keep up with any of this from about 1988 (MS-DOS 4.0) to 2016 (WSL 1), so it has a whole lot of catching up to do. I predict that they'll never rival the likes of xterm given their history of incomplete support of standards developed elsewhere.

…And all this happens in a world that has the comprehensive vttest program in it, so we can all agree on what is "correct." A subset test covering the cases I care about (§1-4, 11.5, 11.6, and 11.8.3 in vttest 2.7) results in a score of 104 on macOS 10.15's Terminal by my reckoning but 69 on Windows Terminal Preview Release. Microsoft lost most of those points by not supporting the 132-column and double-width modes, which lack you may wish to ignore, but there are several unrelated tests it also fails. I assume the "stable" version of Windows Terminal scores the same or worse.

EDIT 1: I installed WSL 2 on a Windows 10 box I happened to have handy — not a normal condition for me — with VSCode and Microsoft's WSL Remote extension and repeated the above test. I get a score of about 61. There were actually a few things it did right relative to both Windows Terminal and Apple's Terminal.app, but on balance, it's missing features relative to them. Oh, and that score only applies if you're charitable and ignore its lack of blink support and dark/light mode switching. It'd be roughly in the 40s if you insisted on those features working.

Windows Terminal and VSCode Terminal aren't the only option on Windows. There are third-party alternative terminals such as PuTTY/MinTTY and SecureCRT. This doesn't solve your VSCode problem, but it once again shows that many different organizations can score high on vttest while Microsoft lags.

Bottom line: I completely fail to be stunned by the news that there's yet another piece of Microsoft software not getting this right.


P.S. to those unclear on what any of this has to do with SQLite: if you use a buggy terminal with SQLite, you can expect buggy terminal-related behavior from SQLite.


EDIT 2: All of these tests were done with the terminal resized to 80x24 to give it the best chance of earning points on each test. Most terminal emulators will cope reasonably well when run larger than this, but you can pretty much guarantee bad behavior when you run any X3.64 family terminal smaller than this. I'm posting this edit because I expect VSCode Terminal to regularly be run smaller than 80x24, on the height at least, so that's a further reason to expect brokenness here.

(12.1) By curmudgeon on 2021-02-19 18:47:18 edited from 12.0 in reply to 11.3 [link] [source]

Thanks for that explanation Warren. I googled xterm wondering if it could be used for the vscode terminal and came across this page

https://www.npmjs.com/package/vscode-xterm

which seems to suggest vscode is using xterm.js ????

(13) By Warren Young (wyoung) on 2021-02-19 18:54:25 in reply to 12.1 [link] [source]

I expect the name xterm.js is a reference to xterm-the-protocol — being one of the ANSI X3.64 family mates — as opposed to xterm-the-program, the reference standard for that protocol, maintained by the creator of vttest.

xterm-the-program should score 100% on vttest, but I'm not going to bother trying it.