SQLite Forum

Sending output to a pipe
Login

Sending output to a pipe

(1) By Harald Hanche-Olsen (hanche) on 2021-04-24 12:56:44 [link] [source]

It has clearly been a while since I tried this, so I was quite surprised when

.once |less

fired up the less command immediately, rendering me unable to execute the sql command I had intended to. I had expected sqlite to wait for me to type in a query, then execute the query, and finally firing up the command and piping the query output into it.

Am I doing this wrong?

(2) By Larry Brasfield (larrybr) on 2021-04-24 13:37:56 in reply to 1 [link] [source]

You're not doing it wrong, but your expectation is amiss. The "less" utility displays it input stream and grabs console input and writes all over the console screen.

(3) By Harald Hanche-Olsen (hanche) on 2021-04-24 14:49:35 in reply to 2 [link] [source]

Indeed, I know and expect that less does all that, but it would not have been a problem if the sqlite shell would wait until there is some output to display before invoking the external command. It would also have to wait for the command to finish before printing a new prompt and trying to read user input once more.

But never mind: Now that I know it is not expected to work, I can find other ways.

I imagine others might expect the same behaviour I did, though, so perhaps it could deserve a mention in the documentation.

(4) By ThanksRyan on 2021-04-24 20:13:09 in reply to 3 [link] [source]

I imagine others might expect the same behaviour I did, though, so perhaps it could deserve a mention in the documentation.

What shall the documentation say?

$ sqlite3 mydatabase.db | less This doesn't wait for me to enter some commands before running less. Is this a problem with SQLite?

(7) By Larry Brasfield (larrybr) on 2021-04-24 23:29:13 in reply to 3 [source]

What .once and .output do, when they are told to create a piped-into process (rather than a file), is start the specified process in conjunction with creation of a pipe, output of which becomes the new process' stdin and input of which receives the SQLite shell output for some command(s). Meanwhile, the shell does not give its stdin over to the piped-into process or begin processing input in any different way than before.

I can see and semi-agree that it might be possible for the .once code to only start the piped-into process after collecting input for the next metacommand or query, then waiting for that process to finish before trying to collect any more input. That might even do what you expect when you are running the shell interactively. To make .output still work, it would be necessary to collect and queue up all subsequent metacommands and queries up to an ".output stdin" metacommand. I can imagine making that work, too. (I must confess to having imagined much more than ever came to be.)

Once that was working, the behavior would have to be documented carefully because it would be more complex than would usually be expected, and more prone to creation of surprise or confusion.

I'm not sure that the present behavior needs more documentation. I certainly do not see a need for any explanation of how piped-into processes should be expected to interact with the shell when they go beyond simply accepting input on the stream they are given. (The possibilities and combinations are too boundless.) However, I can see that maybe more words might clarify the fact that piped-into processes are created when a meta-command specifies such. The trick would be to add something not evident to those who know what pipes are and what they do.

(8) By Harald Hanche-Olsen (hanche) on 2021-04-25 13:03:39 in reply to 7 [link] [source]

I agree that it is not reasonable to expect .output to behave any differently than it currently does; that would be way too complicated. The thing is, I don't think I have ever used .output, but I use .once regularly. And when you don't connect the two commands in your head, it is very easy to fall into the trap of assuming that .once will wait before actually running the named command. Clearly, that is what happened to me. (I do know about pipes and what they do; so here is at least one data point suggesting it is not evident.)

Here is one suggestion for an addition to the documentation:

The given command is started immediately. Beware of specifying a command that will try to interact with the terminal.

This should provide sufficient warning for knowledgeable users. But perhaps it is too cryptic for less sophisticated users.

(5) By Kees Nuyt (knu) on 2021-04-24 22:05:53 in reply to 1 [link] [source]

It is still very useful in scripts:

sqlite3 -bail test.sqlite <<EOSQL
CREATE TABLE a (
  id  INTEGER PRIMARY KEY NOT NULL
, tx  TEXT
);
INSERT INTO a VALUES (1,'one');
INSERT INTO a VALUES (2,'two');
INSERT INTO a VALUES (3,'three');
.once |less
SELECT * FROM a;
EOSQL

(6) By Keith Medcalf (kmedcalf) on 2021-04-24 22:09:57 in reply to 1 [link] [source]

Use the solution used by single-tasking Operating Systems that cannot run multiple processes simultaneously -- ie, where a "pipe" is implemented by running the first program and redirecting the "pipe" output to a file, then after than is finished, run the second program with its input attached to the file.

.once tempfile.txt
[... your command goes here ...]
.system less tempfile.txt