CLI - Length of paste
(1.1) By Aask (AAsk1902) on 2023-06-06 09:15:39 edited from 1.0 [link] [source]
Using v3.42 on Windows 11, when I paste a long SQL statement:
a. it gets truncated: only the first 255 characters are pasted
b. I cannot type from the end of the line either.
Is this a limit in the CLI?
(2) By Warren Young (wyoung) on 2023-06-06 13:20:10 in reply to 1.1 [link] [source]
You already started a thread on this topic last year and got suitable workarounds.
(3.1) By Aask (AAsk1902) on 2023-06-06 17:51:30 edited from 3.0 in reply to 2 [link] [source]
Apologies; I'd forgotten (and should have searched).
From 4094 to 255 characters - seems regressive. I'll use one of the workarounds.
(4) By Larry Brasfield (larrybr) on 2023-06-06 20:53:12 in reply to 1.1 [link] [source]
Is this a limit in the CLI?
No. The CLI accepts input from stdin without a length limit other than what is imposed by the memory seemingly available1 to the process.
- ^ I say "seemingly available" because when the putatively available memory is obtained via paging and virtual memory trickery, use of such pseudo-memory can slow things to such a crawl that process kill is the best recourse. And even that can become difficult to do.
(5) By Scott Robertson (UdGuru) on 2024-05-07 02:48:48 in reply to 4 [link] [source]
I'm not qualified to agree or disagree but my experience is that for five years I was pasting anything I wanted into an older SQLite cli. When I upgraded to 3.45.1 I could still paste anything into the Windows CMD or Powershell until starting the SQLite cli, then same story as Aask, only 255 characters are pasted and nothing else can be typed. It seems obvious that the SQLite cli would be causing this. Not that I understand your explanation about seemingly available memory, it's over my head.
Thanks for your workaround re: cbecho.exe which I used to answer my related question:
(6.4) By Aask (AAsk1902) on 2024-05-07 08:20:40 edited from 6.3 in reply to 5 [link] [source]
I repeated the exercise with version 3.45.3 and an SQL which is 5,278 characters long.
Now I am prompted:
You are about to paste text that is longer than 5KiB. Do you wish to continue?
Paste Anyway or Cancel
The prompt appears in BOTH SQLite AND at the Command Prompt ; there is no truncation.
Both options work as expected/implied in SQLite and the Command Prompt.
I might be tempted to agree with this but that would imply that SQLite is affecting the Command Prompt - rather unlikely (?).
What is different?
- SQLite - from version 3.42 to 3.45.3
- Windows 11 - several updates applied between 2023-06-06 and today.
(7) By Scott Robertson (UdGuru) on 2024-05-07 09:19:11 in reply to 6.4 [link] [source]
I agree that it's more likely that Windows downgraded its abilities than for SQLite to do such a thing to itself. With Windows it seems like official policy that doing less is progress. But the change didn't occur when I changed from Windows 10 to Windows 11, it might have occurred when I changed from an older SQLite to 3.45.1. But like you say, it could have occurred with one of Windows' updates and I wouldn't have known. With a staff of 3, I don't think the SQLite team has time to mess up its product in the name of progress.
(8) By Warren Young (wyoung) on 2024-05-07 12:11:13 in reply to 7 [link] [source]
I'd be interested to hear if any of this changes when one of you Windows fans tries it under Microsoft's new Terminal, possibly coupled with PowerShell.
Learning that cmd.exe
and its associated Windows Console sucks hasn't been news since about 1993. If this poor behavior persists under MS's latest terminal stuff, though, then we can talk about losing functionality with upgrades.
A related test would be to see what happens on WSL through Terminal. If that's good but PowerShell is bad, well, that would point the finger pretty directly, wouldn't it?
(9) By niklasb on 2024-05-07 12:23:22 in reply to 8 [link] [source]
Just tested in plain old cmd.exe in plain old Windows console on Windows 10.
Had no problems pasting 7600 characters. The limit seems to be around 8000.
(10.3) By Aask (AAsk1902) on 2024-05-07 15:41:23 edited from 10.2 in reply to 8 [source]
Easy way to tell Windows Console and Windows Terminal apart is that they are, respectively, single and multi tab windows.
under Microsoft's new Terminal, possibly coupled with PowerShell.
No SQLite pasting issues when SQLite3.exe started either way.
Windows Console
SQLite: The 256 character limit still applies with both user- and administrator privileges.
Command Prompt: No 256 character limit.
Looks like an SQLite3 issue.
PS:
- I am using Microsoft Windows [Version 10.0.22631.3447] (Windows 11)
- I am not going to install WSL just for this.
(12) By Larry Brasfield (larrybr) on 2024-05-07 16:30:37 in reply to 10.3 [link] [source]
... Looks like an SQLite3 issue.
There is no code in the CLI that limits input beyond the fact that a memory allocation to hold that input may fail. The CLI is entirely (and blissfully) ignorant as to whether input is being pasted in or typed in, and it does nothing differently in the input collection process between console inputs and other stream sources.
The effect you see (and which I confirm) is something happening in the conhost.exe process which supplies a "text I/O window" to the sqlite3.exe process.
(13) By Aask (AAsk1902) on 2024-05-07 18:34:03 in reply to 12 [link] [source]
I can run SQLite3.EXE EITHER within the conhost.exe process (Windows Console) OR within the wt.exe process (Windows Terminal).
The truncation happens ONLY within the conhost.exe process NOT within the wt.exe process.
If:
is something happening in the conhost.exe process which supplies a "text I/O window" to the sqlite3.exe process.
If SQLite3.EXE is dependent on the conhost.exe process, shouldn't the truncation also happen when SQLite3 is running under the wt.exe process?
(14) By Larry Brasfield (larrybr) on 2024-05-08 00:36:33 in reply to 13 [link] [source]
If SQLite3.EXE is dependent on the conhost.exe process,
The CLI knows nothing of conhost.exe or processes running it.
On Windows, long-standing console APIs are used for console detection and console text stream I/O. These are, purportedly, Windows APIs, and they are present and useful whether the console is implemented by conhost.exe or wt.exe.
In either console host, pasting is handled completely outside the purview of the sqlite3.exe process. As I said, sqlite3.exe merely reads input provided via the console without regard for whether it was pasted or typed.1 Either way, it is read by the same console API call(s).
To explain the strange pasting behavior (adequately), it will be necessary to look at factors completely absent from the CLI source.
- ^ There are other ways to feign console input; sqlite3.exe would not distinguish those, either.
(11) By curmudgeon on 2024-05-07 15:38:59 in reply to 8 [link] [source]
I tried pasting a long create table statement into sqlite3 started in a manjaro terminal inside vscode (so via WSL2 - if that's the right terminology).
The CLI didn't show the entire paste. It seemed to show around 255 chars, made up of the contiguous first n lines of the paste (only around 25% of the whole command) but with the bottom line showing the final line of the paste with the terminating semi-colon wrapped round to the start of the line???
That said, after pressing return, the table was created with all the correct fields in place.
(15) By Scott Robertson (UdGuru) on 2024-05-08 02:50:41 in reply to 11 [link] [source]
I tried pasting the long create table command into cmd.exe with the SQLite cli running and only the first 255 chars showed up. There was no semicolon anywhere, and pressing enter didn't run the command, didn't create the table.
(16) By Scott Robertson (UdGuru) on 2024-05-08 03:04:21 in reply to 8 [link] [source]
Thanks for the suggestion to use Terminal. I'd purposely forgotten it exists because years ago when I first tried it, I found it can't be resized larger without its going bonkers. Say for example you're trying to read a table with a lot of columns, so you make the window bigger. The text inside it does not redisplay stretched out across the bigger window, AND the command line moves up into that previous output. You can keep typing, the commands will display in the wrong place and they will work, but it makes me spastic to see MS not bothering to fix obvious defects like this.
HOWEVER, the Terminal is the best, easiest, simplest workaround for my problem (pasting-then-editing large create table commands) because you can paste the old schema of any length, edit it, see it, and it works as expected. But if you want to resize the window, you have to do it before you type anything.
Incidentally, the "broken since 1993" cmd.exe was improved some time ago. You can resize the window and the output will redisplay properly. You can select and copy non-rectangular blocks of text. So yeah... cmd got better and then it did lose functionality, at least on my system which is up-to-date Windows 11.
Thanks for helping this "Windows fan" find a simple way to keep doing what I've been doing for years.