Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | In the query planner, add a heuristic that will reduce the estimated cost of a full table scan for a materialized view or subquery if the full scan is the outer-most loop. This is shown to speed up some queries. |
---|---|
Downloads: | Tarball | ZIP archive | SQL archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA3-256: |
609fbb94b8f01d6792e5941ab23ce041 |
User & Date: | drh 2022-09-01 10:29:02 |
Original Comment: | In the query planner, add a heuristic that will reduce the cost of a full table scan for a materialized view or subquery if the full scan is the outer-most loop. This is shown to speed up some queries. |
References
2023-09-15
| ||
19:51 | Drop support for the view-scan optimization (check-in [609fbb94b8f01d67]) as it was causing multiple performance regressions. In its place, reduce the estimated row count for DISTINCT subsqueries by a factor of 8. (check-in: f911f1c4 user: drh tags: trunk) | |
2023-07-18
| ||
21:06 | Do not use the viewscan optimization on a query that has only a single loop, as the cost adjustments can cause problems for outer queries. Proposed fix for the performance regression reported by forum post 64d36440e473516c. (check-in: 76152ad2 user: drh tags: trunk) | |
2023-01-30
| ||
20:44 | Additional tweaks to the enhancement at [609fbb94b8f01d67] to further reduce the cost estimate for constructing an automatic index on an ephemeral table, in order to resolve the performance problem described by forum post 1d571c0296. (check-in: bf1aae7a user: drh tags: trunk) | |
Context
2022-09-01
| ||
13:51 | Defer deleting a transient SELECT statement associated with a flattening of one arm of a compound SELECT until after the parse has completed. This is a follow-up and enhancement to check-in [6e6b3729e0549de0] in response to an assertion fault reported by Chromium. (check-in: 1c4157c7 user: drh tags: trunk) | |
10:41 | In the query planner, add a heuristic that will reduce the cost of a full table scan for a materialized view or subquery if the full scan is the outer-most loop. This is shown to speed up some queries. (check-in: e3754cc1 user: drh tags: branch-3.28) | |
10:29 | In the query planner, add a heuristic that will reduce the estimated cost of a full table scan for a materialized view or subquery if the full scan is the outer-most loop. This is shown to speed up some queries. (check-in: 609fbb94 user: drh tags: trunk) | |
2022-08-31
| ||
15:04 | Enhance the b-tree page sorting code to ensure that sqlite3PagerRekey() never overloads a page number and uses only the PENDING_BYTE page for temporary storage. (check-in: 50077428 user: drh tags: trunk) | |
Changes
Changes to src/where.c.
︙ | ︙ | |||
3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 | ** better. */ #ifdef SQLITE_ENABLE_STAT4 pNew->rRun = rSize + 16 - 2*((pTab->tabFlags & TF_HasStat4)!=0); #else pNew->rRun = rSize + 16; #endif ApplyCostMultiplier(pNew->rRun, pTab->costMult); whereLoopOutputAdjust(pWC, pNew, rSize); rc = whereLoopInsert(pBuilder, pNew); pNew->nOut = rSize; if( rc ) break; }else{ Bitmask m; | > > > | 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 | ** better. */ #ifdef SQLITE_ENABLE_STAT4 pNew->rRun = rSize + 16 - 2*((pTab->tabFlags & TF_HasStat4)!=0); #else pNew->rRun = rSize + 16; #endif if( IsView(pTab) || (pTab->tabFlags & TF_Ephemeral)!=0 ){ pNew->wsFlags |= WHERE_VIEWSCAN; } ApplyCostMultiplier(pNew->rRun, pTab->costMult); whereLoopOutputAdjust(pWC, pNew, rSize); rc = whereLoopInsert(pBuilder, pNew); pNew->nOut = rSize; if( rc ) break; }else{ Bitmask m; |
︙ | ︙ | |||
4838 4839 4840 4841 4842 4843 4844 4845 4846 4847 4848 4849 4850 4851 | ("---- sort cost=%-3d (%d/%d) increases cost %3d to %-3d\n", aSortCost[isOrdered], (nOrderBy-isOrdered), nOrderBy, rUnsorted, rCost)); }else{ rCost = rUnsorted; rUnsorted -= 2; /* TUNING: Slight bias in favor of no-sort plans */ } /* Check to see if pWLoop should be added to the set of ** mxChoice best-so-far paths. ** ** First look for an existing path among best-so-far paths ** that covers the same set of loops and has the same isOrdered ** setting as the current path candidate. | > > > > > > > | 4841 4842 4843 4844 4845 4846 4847 4848 4849 4850 4851 4852 4853 4854 4855 4856 4857 4858 4859 4860 4861 | ("---- sort cost=%-3d (%d/%d) increases cost %3d to %-3d\n", aSortCost[isOrdered], (nOrderBy-isOrdered), nOrderBy, rUnsorted, rCost)); }else{ rCost = rUnsorted; rUnsorted -= 2; /* TUNING: Slight bias in favor of no-sort plans */ } /* TUNING: A full-scan of a VIEW or subquery in the outer loop ** is not so bad. */ if( iLoop==0 && (pWLoop->wsFlags & WHERE_VIEWSCAN)!=0 ){ rCost += -10; nOut += -30; } /* Check to see if pWLoop should be added to the set of ** mxChoice best-so-far paths. ** ** First look for an existing path among best-so-far paths ** that covers the same set of loops and has the same isOrdered ** setting as the current path candidate. |
︙ | ︙ |
Changes to src/whereInt.h.
︙ | ︙ | |||
644 645 646 647 648 649 650 651 652 | #define WHERE_IN_EARLYOUT 0x00040000 /* Perhaps quit IN loops early */ #define WHERE_BIGNULL_SORT 0x00080000 /* Column nEq of index is BIGNULL */ #define WHERE_IN_SEEKSCAN 0x00100000 /* Seek-scan optimization for IN */ #define WHERE_TRANSCONS 0x00200000 /* Uses a transitive constraint */ #define WHERE_BLOOMFILTER 0x00400000 /* Consider using a Bloom-filter */ #define WHERE_SELFCULL 0x00800000 /* nOut reduced by extra WHERE terms */ #define WHERE_OMIT_OFFSET 0x01000000 /* Set offset counter to zero */ #endif /* !defined(SQLITE_WHEREINT_H) */ | > | 644 645 646 647 648 649 650 651 652 653 | #define WHERE_IN_EARLYOUT 0x00040000 /* Perhaps quit IN loops early */ #define WHERE_BIGNULL_SORT 0x00080000 /* Column nEq of index is BIGNULL */ #define WHERE_IN_SEEKSCAN 0x00100000 /* Seek-scan optimization for IN */ #define WHERE_TRANSCONS 0x00200000 /* Uses a transitive constraint */ #define WHERE_BLOOMFILTER 0x00400000 /* Consider using a Bloom-filter */ #define WHERE_SELFCULL 0x00800000 /* nOut reduced by extra WHERE terms */ #define WHERE_OMIT_OFFSET 0x01000000 /* Set offset counter to zero */ #define WHERE_VIEWSCAN 0x02000000 /* A full-scan of a VIEW or subquery */ #endif /* !defined(SQLITE_WHEREINT_H) */ |
Changes to test/window1.test.
︙ | ︙ | |||
208 209 210 211 212 213 214 | CREATE TABLE t2(x); INSERT INTO t2 VALUES('b'), ('a'); SELECT x, count(*) OVER (ORDER BY x) FROM t1; } {1 1 2 2 3 3 4 4 5 5 6 6 7 7} do_execsql_test 6.2 { | | > < > | 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | CREATE TABLE t2(x); INSERT INTO t2 VALUES('b'), ('a'); SELECT x, count(*) OVER (ORDER BY x) FROM t1; } {1 1 2 2 3 3 4 4 5 5 6 6 7 7} do_execsql_test 6.2 { SELECT * FROM t2, (SELECT x, count(*) OVER (ORDER BY x) FROM t1) ORDER BY 1, 2; } { a 1 1 a 2 2 a 3 3 a 4 4 a 5 5 a 6 6 a 7 7 b 1 1 b 2 2 b 3 3 b 4 4 b 5 5 b 6 6 b 7 7 } do_catchsql_test 6.3 { SELECT x, lag(x) FILTER (WHERE (x%2)=0) OVER w FROM t1 WINDOW w AS (ORDER BY x) } {1 {FILTER clause may only be used with aggregate window functions}} |
︙ | ︙ |
Changes to test/with3.test.
︙ | ︙ | |||
127 128 129 130 131 132 133 | QUERY PLAN |--MATERIALIZE c | |--SETUP | | |--SCAN CONSTANT ROW | | `--SCALAR SUBQUERY xxxxxx | | `--SCAN w2 | `--RECURSIVE STEP | | | | 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | QUERY PLAN |--MATERIALIZE c | |--SETUP | | |--SCAN CONSTANT ROW | | `--SCALAR SUBQUERY xxxxxx | | `--SCAN w2 | `--RECURSIVE STEP | |--SCAN c | `--SCAN w1 |--SCAN c |--SEARCH w2 USING INTEGER PRIMARY KEY (rowid=?) `--SEARCH w1 USING INTEGER PRIMARY KEY (rowid=?) } do_execsql_test 4.0 { WITH t5(t5col1) AS ( |
︙ | ︙ |