SQLite Forum

A segmentation fault in SQLite latest release build
Login
Hello developers,

We found a test case causing a segmentation fault in [SQLite latest release build (3.36.0)](https://sqlite.org/releaselog/3_36_0.html).

The test case is as follows:

```
CREATE VIRTUAL TABLE rt0 USING rtree(c0,c1,c2 INT);
INSERT INTO rt0(c2) VALUES(0);
SELECT * FROM (SELECT rt0.c2,rt0.c1 FROM rt0 UNION ALL SELECT rt0.c1,rt0.c1 FROM rt0) LEFT JOIN (SELECT * FROM (SELECT (SELECT max(sum(rt0.c1),rank() OVER (ORDER BY ra0.c2)) FROM rt0 AS ra0) AS ca1 FROM rt0) WHERE ca1=0);
```

which causes:

> sqlite3: sqlite3.c: RtreeNode *rtreeNodeOfFirstSearchPoint(RtreeCursor *, int *): Assertion `pCur->bPoint || pCur->nPoint' failed.

when compiling with option `--enable-debug`, and segmentation fault when the option is disabled.

Here is the gdb debugging information:

```
#0  0x00005648d420ed44 in rtreeNodeOfFirstSearchPoint (
    pCur=pCur@entry=0x5648d5dc6490, pRC=pRC@entry=0x7ffe9ddf6894)
    at sqlite3.c:193706
#1  0x00005648d420efdd in rtreeColumn (cur=0x5648d5dc6490, ctx=0x7ffe9ddf6a60, 
    i=1) at sqlite3.c:193981
#2  0x00005648d41e2fbb in sqlite3VdbeExec (p=0x5648d5dbf550) at sqlite3.c:94169
#3  0x00005648d41e7410 in sqlite3Step (p=0x5648d5dbf550) at sqlite3.c:84861
#4  sqlite3_step (pStmt=0x5648d5dbf550) at sqlite3.c:19382
#5  0x00005648d414797c in exec_prepared_stmt (pArg=0x7ffe9ddf6f10, 
    pStmt=0x5648d5dbf550) at shell.c:14164
#6  0x00005648d414c1c1 in shell_exec (pArg=<optimized out>, 
    zSql=<optimized out>, pzErrMsg=0x7ffe9ddf6d28) at shell.c:14473
#7  0x00005648d414d946 in runOneSqlLine (p=0x7ffe9ddf6f10, 
    zSql=0x5648d5dc8130 "SELECT * FROM (SELECT rt0.c2,rt0.c1 FROM rt0 UNION ALL SELECT rt0.c1,rt0.c1 FROM rt0) LEFT JOIN (SELECT * FROM (SELECT (SELECT max(sum(rt0.c1),rank() OVER (ORDER BY ra0.c2)) FROM rt0 AS ra0) AS ca1 FR"..., in=0x0, 
    startline=3) at shell.c:21449
#8  0x00005648d4158cf4 in process_input (p=0x7ffe9ddf6f10) at shell.c:21549
#9  0x00005648d4133d59 in main (argc=<optimized out>, argv=0x7ffe9ddf8238)
    at shell.c:22350
```

Bisecting shows the problem may be related to [check-in c510377b0b](https://sqlite.org/src/info/c510377b0b).

Hope the problem will be handled properly :)