SQLite Forum

SQLite3 shell doing math operation on parameter substitution
Login
> My comment was about the need to escape the backslash in the context of your shell script where the usual expansion within double quotes is done on what you forumulate as the .parameter command.

Let leave my script out of discussion, Im pretty sure my code works as expected and passing exact values to sqlite3. I showed test cases in interactive mode of sqlite3 which show the issue.

> The parsing under discussion is occurring exactly as intended and documented, and as it has been done for so many years that we should expect to break many users' scripts if it is changed now

Me too, personally I hate to change any behavior of programs that been deployed in production, but I didn't asked to change anything. I just reporting bug/feature issue that isn't documented and can be **dangerous** (what about if malicious user will supply complicated, long math formula that might trigger internal engine to choke). IMHO, there in documentation no any clue that shell will try to **evaluate** user supplied value, especially **enclosed in quotes**. If some behavior (especially, - value evaluation behind scene) isn't documented, then it looks to me as a bug, even if it is very advanced behavior.

[Rules for "dot-commands"](https://sqlite.org/cli.html#rules_for_dot_commands_) says:

> Text bounded by a pair of single-quotes or double-quotes is treated as a single argument, with the quotes stripped.

and that's exactly what one would expect, - strip a quotes and put enclosed content as is into key/value store.

Read documentation further:
 
> The dot-commands are interpreted by the sqlite3.exe command-line program, not by SQLite itself.

where clearly said: **"not by SQLite itself."**, but who then doing arithmetic calculation if not SQLite? I took a quick look at source code that parsing .dot commands and didn't found any calculations. Anyway, nowhere in documentation says that shell will **evaluate** supplied value, that's why such program behavior IMHO can be called as a bug. 

> Your alleged bug is about a scenario where that parsing is leading to the result you report but did not expect.

Call me nitpicker, but parsing != evaluation, it is two different subsequent steps, first parse, then evaluate, which is comes as surprise.


> Very little about the quoting rules could be fairly called "intuitive", but it is documented where and as I cited earlier in section 4. 

It is probably my bad English, if I still can't explain that the problem isn't with quoting, but with evaluation. If it is says **parameter key value** then the **value** must be a raw value, as is. There no in documentation anything that it says that value **can be a math expression** and this expression will be automatically evaluated.

>  I think the time has come for you to read it, carefully.

Larry, I believe I already remembered this section 4 as a poem, but if I missed there explanation regarding **value evaluation**, I will be more than appreciated if you pointing me there.


Regarding "fixing" a problem. IMHO, first of all documentation must be clearly stated, that a **value** in .parameter will be **analyzed, evaluated and modified if calculated**

To avoid breaking user's scripts who found this undocumented behavior as a feature, they can still use **set** command of .parameter as is now, but there might be implemented another .parameter command, for example **"assign"**, that doing true assignment of raw value to a key (actually exactly what prepared statement must do). IMO the shell can simply threat right hand part of command line as a raw value that beginning right after first space behind key name, up to the end of line and put it as blob to a store. Something like:

<strong>

```
echo "${line}" | sed -r 's/^\.para?m?e?t?e?r?\s+set\s+([@\$]*[A-Za-z_]+[0-9]*|\?[0-9]*)\s//g'
```

</strong>

> Either way it becomes one of the multi-layered quoting tangles that sh programmers love or dread.

I don't agree, the code below can live without ugly escaping and can be cooked on fly if need:

<strong>

```
#!/bin/bash

someVar="aaa 'bbb' ccc"

cat << 'EOF' >test.file.txt
.param set @key2 {"key1":"value1", {"key2":"value2"}
.param set ? {"key1":"value1", {"key2":"value2"}
.param set ?1 {"key1":"value1", {"key2":" ' ' ' ' "}

\t \n
${someVar}

EOF


```

</strong>

If one would check content of **`test.file.txt`** then he/she will find that there no need for escaping. Instead of writing to a file, one can pipe heredoc data to any program as a stream of literal bytes. You can even place inside of heredoc block even raw bytes 0x01, 0x02... and shell will happily write it to a file.

Larry, I really want you to understand that I don't want to offend anyone for sure. If the word "bug" is unwelcome, Im fine with it as long as the issue resolved. As I said, English isn't my native language, so I trying to be more expressive in attempt to describe subject. If I touched some red zones, - it isn't on purpose for sure. Let me know if it is.

I just love SQLite3 a lot and want this tool be better and better, so I hope we are on the same page.