CLI: Is there a limit to the length of what can be pasted?
(1) By Aask (AAsk1902) on 2022-12-13 17:47:52 [link] [source]
Using the 32-bit CLI on Windows, when I paste in the CLI text/script that is longer than 4094 characters, it truncates at 4094 characters. Does this limit apply by design or am I doing something odd?
(2) By Keith Medcalf (kmedcalf) on 2022-12-13 18:26:47 in reply to 1 [link] [source]
This is a limitation of the Windows Console input stream buffer.
The limitation used to be 255 characters. I suppose someone at Microsoft recently realized that Bill was wrong when he posited that 255 (256 with the zero terminator) should be enough for anyone.
(7) By Harald Hanche-Olsen (hanche) on 2022-12-14 16:02:53 in reply to 2 [link] [source]
But nobody will ever need more than 64 kilobytes of RAM, right? Or was it 56?
(3) By Donald Griggs (dfgriggs) on 2022-12-13 18:40:16 in reply to 1 [link] [source]
Aask, you're encountering a limit on the length of a particular line rather than a limit for a multi-line script, is that right?
FWIW, I just succeeded in pasting a script of 68,002 bytes at the CLI prompt.
Win 10, version 1809. Sqlite3.exe 3.39.4 as released, 32-bit version.
(4.1) By Aask (AAsk1902) on 2022-12-13 20:05:38 edited from 4.0 in reply to 3 [link] [source]
... you're encountering a limit on the length of a particular line ...
Yes. No issues with multi-line scripts.
(5) By Simon Slavin (slavin) on 2022-12-14 13:04:16 in reply to 1 [link] [source]
I believe that Windows now has two different native shells. Try it in the other shell.
I can tell you that there's nothing in the SQLite source code that limits things to 4,096 characters (or 4,095, or anything like that).
(6.1) By Larry Brasfield (larrybr) on 2022-12-14 16:40:25 edited from 6.0 in reply to 5 [link] [source]
Confirming Simon's assertion regarding CLI line-length limitation in two ways:
The following line, without any characters elided, can be pasted into the CLI1 or interpreted by it via its .read command:
/* #[8190 characters elided]# */ SELECT 'overly commented query';
, without any strangeness, misbehavior or complaint.The CLI's code for reading input lines uses a dynamically allocated buffer which it grows as necessary to accommodate such ridiculous input.
There are limits, of course. Some length of such absurd input lines is going to produce an out-of-memory error when the CLI attempts to grow its line buffer. It would be boring and pointless to find where that limit is.
- ^ Here, "can be pasted" means without hinderance from the platform. Some platforms, notably those named after certain building openings, impose pasting length limits at their stock console.
(9) By Keith Medcalf (kmedcalf) on 2022-12-14 16:55:12 in reply to 6.1 [link] [source]
Actually, the limitation is not the "console application". It is the internal Windows input line buffer and is quite independent of the application that is attached to the "spigot end" of the Windows input queue. It also applies to "pasted input" into a hooey-gooey field as well. The limitation is at a lower level than the "application". It is an inherent limitation of the Operating System.
Unix-like systems may impose limits at the application level (none exist at the input stream level).
(10) By Larry Brasfield (larrybr) on 2022-12-14 21:37:17 in reply to 9 [link] [source]
Actually, the limitation is not the "console application".
I never claimed (or believed or implied) that the limitation arises in "the console application". That term, as commonly used, refers to an executable which is marked as requiring a console server process to run in tandem with the client process running the said executable.
It is the internal Windows input line buffer and is quite independent of the application that is attached to the "spigot end" of the Windows input queue.
I'm puzzled by what "the internal Windows input line buffer" is supposed to mean. I am quite sure that there is no such thing shared among applications.
I am also sure that the console processes1 are where the pasting truncation is happening. It is those processes which are called upon to respond to GUI events and, for some sequences of such, collect clipboard text content and feed it into the the standard input stream for the hosted "console application".
It is those "stock" console server processes which are electing to truncate clipboard text as it is fed to their client process. It would be perfectly possible to write a less penurious console host which would feed any amount of text it could grab from the clipboard API to its client "console application". But the stock ones do not. As I said.
I think that when you seek to engage in corrective education, you need to respond to what is actually said rather than a hasty misinterpretation.
- ^ The stock consoles on Windows systems today are running instances of one or two Windows applications, the legacy conhost.exe or the newer OpenConsole.exe .
(11) By Keith Medcalf (kmedcalf) on 2022-12-14 21:54:02 in reply to 10 [link] [source]
I am also sure that the console processes1 are where the pasting truncation is happening. It is those processes which are called upon to respond to GUI events and, for some sequences of such, collect clipboard text content and feed it into the the standard input stream for the hosted "console application".
No. The limitation is on anything that reads "line input" from the Windows input queue. You will find the exact same "block" limitation when copying and pasting into, for example, Word, Excel, PowerPoint, Notepad, and on and on. Multiple blocks (lines) can be "pasted" but there is a limitation on the size of each "block".
These are not "console" applications which run under the "Windows input queue" to "normal file stream I/O" adaption layer known colloquially as the "console host", but rather native hooey-gooey applications.
(12.1) By Larry Brasfield (larrybr) on 2022-12-15 00:34:37 edited from 12.0 in reply to 11 [link] [source]
(Edited to save backslashes from death by markup.)
If you run the following1 from an OS shell:
perl -e "print '/', '*'x99999, '/ select \'long line\';';" | clip
, you will find that you can then paste a very long "line" into any Windows GUI application that does not itself elect to ignore part of the block. Notepad is one such. Those would include the console host apps if they had been written with fewer stingy assumptions.
These are not "console" applications which run under the "Windows input queue"
I do not know what you are going on about here. The message queues which GUI applications rely upon are per-process. You cannot speak of "the input queue". But that does not matter. Clipboard content never reaches message queues under normal circumstances. (One could make that happen using API calls.) An application which learns of available clipboard content and a user's desire to access it uses API calls to get that content. And as you can see, by running the above code and typing control-V into a notepad text window, no "block" limitation or other hooey is involved. That little query will show up at the end.
- ^ These are a well-known application and a little utility provided by Windows in its .../system32 directory.
(8) By Keith Medcalf (kmedcalf) on 2022-12-14 16:16:52 in reply to 5 [link] [source]
Actually, no. Windows only has one input stream line buffer handler. The "application" on the other end of the input stream handler is immaterial.
The "paste line limit" in Windows 10 22H2 is 4096. This is irrespective of the application that is reading the line buffered console input stream.
(13.2) By Larry Brasfield (larrybr) on 2022-12-15 03:44:51 edited from 13.1 in reply to 1 [source]
(Edited to mention a nice little tool for "pasting" clipboard text to stdout and how to get it as an executable and use it.)
[Clipboard truncation woes ...] Does this limit apply by design or am I doing something odd?
Yes and no. Or no and yes.
The limit is coded into certain programs that ship as part of the "Windows (tm)" operating system. It may be a stretch to call this "by design"; more likely it bespeaks a lack of design and the imagination to see where more design is needed.
You are doing something a little odd. To get it done, you can enter this into the CLI:
.read "|powershell -Command Get-Clipboard"
. There are less clunky ways to accomplish the same thing, but not without acquiring, creating or building a clipboard-text-to-stdout tool.1 The need for this is common enough that Powershell has an alias for it, "gcb".
- ^ At GitHub, G. Adam Stanislav has made his cbecho.c available. It compiles cleanly with modern tools, and the (19 month old) version I examined has no malware or goofy code in it. Anybody with a Window-targetting C compile/link installation can build an .exe from this single source. The .exe from that source is easy to use from the SQLite CLI.2
- ^
Example usage: .read "|cbecho -c"
This will read absurd amounts of text, without console-imposed limits, so be careful.