SQLite User Forum

Operator precedence bug
Login

Operator precedence bug

(1) By Rico Mariani (rmariani) on 2022-05-31 05:49:04 [link] [source]

I wrote about this some time ago but I never quite distilled it to the basics. The issue is that "IS TRUE" and "IS FALSE" are not their own unary operators but are instead implemented as special cases of "IS"

This is the minimal repro that shows what goes wrong:

PostgreSQL 9.3 (correct)

select null is true || 'x';
falsex

SQLite 3.31 (incorrect)

select null is true || 'x';
0

(2) By Mark Brand (mabrand) on 2022-05-31 11:54:06 in reply to 1 [link] [source]

You are right to point out this pitfall, but "bug" is probably not the right word for documented features.

As documented, "||" precedes "IS". "IS TRUE", "IS NULL", etc are not operators but expressions containing the operator "IS".

https://sqlite.org/lang_expr.html

The following are equivalent in SQLite:

  select null is true || 'x';
  select null is (true || 'x');
  select null is (0 || 'x');
  select null is '0x';
  select '0x' is null;

For better or for worse, current application depend on the status quo.

The difference from PostgreSQL probably deserves mentioning in "Quirks, Caveats, and Gotchas".

https://sqlite.org/quirks.html

(3) By Gunter Hick (gunter_hick) on 2022-05-31 12:36:48 in reply to 2 [link] [source]

note that there is an operator ISNULL that performs as expected by the OP:

asql> select true is null || 'X';
true is null || 'X'
-------------------
0
asql> select true isnull || 'X';
true isnull || 'X'
------------------
0X

(4) By Rico Mariani (rmariani) on 2022-06-02 05:10:37 in reply to 2 [link] [source]

The docs say this:

The boolean identifiers TRUE and FALSE are usually just aliases for the integer values 1 and 0, respectively. However, if TRUE or FALSE occur on the right-hand side of an IS operator, then they form new unary postfix operators "IS TRUE" and "IS FALSE" which test the boolean value of the operand on the left.

This makes the situation worse. They most certainly do not form new unary postfix operators!

(5) By Larry Brasfield (larrybr) on 2022-06-02 14:50:22 in reply to 4 [link] [source]

... if TRUE or FALSE occur on the right-hand side of an IS operator, then they form new unary postfix operators "IS TRUE" and "IS FALSE" which test the boolean value of the operand on the left.

They most certainly do not form new unary postfix operators!

Do you have a set of expressions which disprove that assertion in the doc?

Incidentally, the operator precedence table does not yet reflect that assertion, which anomaly I plan to correct soon. So I am quite interested in your counter-example.

(6) By anonymous on 2022-06-02 17:12:46 in reply to 5 [link] [source]

SELECT 2 IS TRUE, 2 IS (TRUE), 2 IS +TRUE;
+-----------+-------------+------------+
| 2 IS TRUE | 2 IS (TRUE) | 2 IS +TRUE |
+-----------+-------------+------------+
| 1         | 1           | 0          |
+-----------+-------------+------------+

If IS TRUE had been an actual unary postfix operator, the second expression would have been equivalent to the third one rather than to the first one.

What actually happens is this:

  1. Syntactically, only the binary IS operator exists.
  2. Semantically, the right subtree of the IS operator is checked to see if it consists of a single TRUE or FALSE keyword.
  3. TRUE and (TRUE) generate the exact same expression tree.
  4. TRUE and +TRUE generate different expression trees.

(7) By Keith Medcalf (kmedcalf) on 2022-06-02 18:08:55 in reply to 6 [link] [source]

The construct x IS TRUE is syntactic sugar for x != 0
The construct x IS FALSE is syntactic sugar for x == 0

IS TRUE and IS FALSE are NOT infix operators.

Without the IS preface, the True is the contstant value 1 and False is the constant value 0.

Were IS TRUE and IS FALSE were infix operators, then the result is simply the value of the NZ flag after loading the value into the accumulator.

(13) By Rico Mariani (rmariani) on 2022-06-07 00:00:43 in reply to 7 [link] [source]

If you like postfix IS TRUE can be sugar for IFNULL(x != 0, 0). But the above is incorrect.

If you care to experiment, you'll find that there is no wide agreement about whether IS NULL is a postfix unary operand or part of a binary IS operand. SQLite, PostgreSQL and MySQL give differing results. I couldn't get Oracle or SQL Server to take my expressions at all...

But in any case the SQLite docs had no weakness with regard to IS NULL--they describe the implementation correctly which is all we can ask for.

(9) By Larry Brasfield (larrybr) on 2022-06-03 15:54:14 in reply to 4 [link] [source]

A reexamination of that language and the implementation has shown that {IS, IS NOT} paired with {TRUE, FALSE} is better not considered a unary postfix operator. This has led to an alteration of the language you quote. There are undoubtedly more comprehensive and wordier explanations, but this minor feature is an optional nicety1 whose full description would be a distraction from the more important information about expressions.


  1. ^ The operator is a shortcut for a slightly longer expression involving ISNULL variants.

(12) By Rico Mariani (rmariani) on 2022-06-06 23:31:32 in reply to 9 [source]

I think that's a nice improvement and if someone really cares deeply about this (like if they have codegen that targets SQLite) it will tip them off that they need an extra set of parens to complete the postfix op illusion.

(8.1) By Keith Medcalf (kmedcalf) on 2022-06-02 18:41:09 edited from 8.0 in reply to 1 [link] [source]

It would depend if null is true should return False (0), or NULL.

IMHO, select null is true should be NULL, not 0 (false).

It should NOT be processessed as select null is not 0, though processing as select null != 0 is correct.

SQL has tri-state logic. It is very complicated.

IMHO both PostgreSQL and SQLite3 are incorrect and IS TRUE and IS FALSE should not be used (because they have incorrect results) and only the correct expression != 0 for IS TRUE and == 0 for IS FALSE should be used.

(10) By Rico Mariani (rmariani) on 2022-06-06 23:28:08 in reply to 8.1 [link] [source]

IMHO, select null is true should be NULL, not 0 (false).

You could try to sway the standards bodies to change it but I don't think they're wrong. The point of IS TRUE vs. != 0 is to normalize null. IFNULL(x != 0, 0) is hard on the eyes and it does come up a lot. At this point every system agrees that x IS TRUE and x IS FALSE never return null.

IMHO both PostgreSQL and SQLite3 are incorrect and IS TRUE and IS FALSE should not be used (because they have incorrect results) and only the correct expression != 0 for IS TRUE and == 0 for IS FALSE should be used.

I don't think the standard leaves room for interpretation on this. IS TRUE and IS FALSE are not tristate. Sqlite's issues stem from the fact that it doesn't actually have IS TRUE or IS FALSE operators, instead it uses its own (not exactly standard) "IS" with a special case of the boolean true or false as the right operand but that results in binding strength that is not standard. Hence the bug.

In practice this is a very minor issue because it takes skill to craft an example where SQLite's (expeditious?) choice makes a difference compared to the perfect choice.

However, as I've written previously the SQLite docs are not at all clear on what SQLite is actually doing. Maybe quirks is the best place to put this. It is worth it to be clear that SQLite only approximates unary postfix IS TRUE.

IMHO, it would not be good for SQLite to take a position of "don't use this well specified standard operator that is in our grammar" especially if the only reason to not use it is that in very quirky cases that almost never happen SQLite gets it wrong.

(11) By Rico Mariani (rmariani) on 2022-06-06 23:29:02 in reply to 10 [link] [source]

The docs seem to have been cleared up which I think is the only thing that was really necessary here.