/ View Ticket
Login
Ticket UUID: 787fa716be3a7f650cac1b1413f12f95f5e7639d
Title: Assertion fault when multi-use subquery implemented by co-routine
Status: Fixed Type: Code_Defect
Severity: Severe Priority: Immediate
Subsystem: Unknown Resolution: Fixed
Last Modified: 2019-05-19 17:01:13
Version Found In: 3.25.3
User Comments:
drh added on 2018-11-08 15:06:12:

The following SQL gives an assertion fault:

CREATE TABLE artists (
  id integer NOT NULL PRIMARY KEY AUTOINCREMENT,
  name varchar(255)
);
CREATE TABLE albums (
  id integer NOT NULL PRIMARY KEY AUTOINCREMENT,
  name varchar(255),
  artist_id integer REFERENCES artists
);
INSERT INTO artists (name) VALUES ('Ar');
INSERT INTO albums (name, artist_id) VALUES ('Al', 1);
SELECT artists.*
FROM artists
INNER JOIN artists AS 'b' ON (b.id = artists.id)
WHERE (artists.id IN (
  SELECT albums.artist_id
  FROM albums
  WHERE ((name = 'Al')
    AND (albums.artist_id IS NOT NULL)
    AND (albums.id IN (
      SELECT id
      FROM (
        SELECT albums.id,
               row_number() OVER (
                 PARTITION BY albums.artist_id
                 ORDER BY name
               ) AS 'x'
        FROM albums
        WHERE (name = 'Al')
      ) AS 't1'
      WHERE (x = 1)
    ))
    AND (albums.id IN (1, 2)))
));

This problem was reported on the mailing list by Jeremy Evans.


drh added on 2018-11-08 19:27:38:

This appears to have nothing to do with window functions. The use of window functions in the example above merely disables the query flattener. The problem can also be seen in the following query:

CREATE TABLE t1 (a);
CREATE TABLE t2 (b);
CREATE TABLE t3 (c);
CREATE TABLE t4 (d);
INSERT INTO t1 (a) VALUES (104);
INSERT INTO t2 (b) VALUES (104);
INSERT INTO t3 (c) VALUES (104);
INSERT INTO t4 (d) VALUES (104);

-- Disable the query-flattener
.testctrl optimizations 0x01

SELECT *
FROM t1 CROSS JOIN t2 ON (t1.a = t2.b) WHERE t2.b IN (
  SELECT t3.c
  FROM t3
  WHERE t3.c IN (
    SELECT d FROM (SELECT d FROM t4) AS innermost WHERE innermost.d=104
  )
);

The query above fails in all versions of SQLite from 3.8.11 and later. The check-in that causes the failure is [020b8b106fc8f840f6b506e1c9c3b] on 2015-05-29


drh added on 2018-11-08 20:49:18:

Here is another test case that does not require manually disabling the query flattener:

CREATE TABLE t5(a, b, c, d);
CREATE INDEX t5a ON t5(a);
CREATE INDEX t5b ON t5(b);
CREATE TABLE t6(e);
INSERT INTO t6 VALUES(1);
INSERT INTO t5 VALUES(1,1,1,1), (2,2,2,2);
SELECT * FROM t5 WHERE (a=1 OR b=2) AND c IN (
  SELECT e FROM (SELECT DISTINCT e FROM t6) WHERE e=1
);

The failure in this test case also goes back to check-in [020b8b106fc8f840f6b506e1c9c3bc75676daec3] on 2015-05-29.


drh added on 2018-11-08 22:03:39:

Here is a variant of the second test case that does not require the manual disabling of the query flattener:

CREATE TABLE t1 (a); INSERT INTO t1 (a) VALUES (104);
CREATE TABLE t2 (b); INSERT INTO t2 (b) VALUES (104);
CREATE TABLE t3 (c); INSERT INTO t3 (c) VALUES (104);
CREATE TABLE t4 (d); INSERT INTO t4 (d) VALUES (104);
SELECT *
FROM t1 CROSS JOIN t2 ON (t1.a = t2.b) WHERE t2.b IN (
  SELECT t3.c
  FROM t3
  WHERE t3.c IN (
    SELECT d FROM (SELECT DISTINCT d FROM t4) AS x WHERE x.d=104
  )
);


drh added on 2018-11-09 13:39:45:

Here is another test case that does not make use of the IN operator. The common factor is now the use of co-routines to implement a subquery and then evaluating that subquery more than once.

CREATE TABLE t1(a1, a2, a3);
CREATE INDEX t1a2 ON t1(a2, a1);
CREATE INDEX t1a3 ON t1(a3, a1);
CREATE TABLE t2(d);
INSERT INTO t1 VALUES(1, 1, 1), (2, 2, 2);
INSERT INTO t2 VALUES(22);
SELECT * FROM t1 WHERE (a2=1 OR a3=2) AND a1 = (
  SELECT d FROM (SELECT DISTINCT d FROM t2) WHERE d=22
);

This new test case demonstrates that checkin [8d663bfaaa4656c4f4ff1] is not a solution to the problem.


mrigger added on 2019-05-18 21:22:10:

In the latest version of SQLite (3.28.0), I could use the following statement sequence to cause a segfault:

CREATE TABLE t0 (c0, c1, PRIMARY KEY (c0, c1)); CREATE TABLE t1 (c0); INSERT INTO t1 VALUES (2); SELECT * FROM t0, t1 WHERE (t0.c1 >= 1 OR t0.c1 < 1) AND t0.c0 IN (1, t1.c0) ORDER BY 1;


mrigger added on 2019-05-19 17:01:13:

Note that I sent the test case that triggers the segfault via a personal email to the SQLite developers. The bug was fixed by check-in [778b1224a318d013] shortly after and before I attached the bug report as part of this ticket.