/ File History
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

History of ext/fts3/fts3Int.h

2019-11-17
02:41
Better detection of corruption in the %_stat and %_docsize shadow tables of FTS3. file: [7201d419] check-in: [1e449687] user: drh branch: trunk, size: 25137
2019-11-16
23:47
More improvements to shadow table corruption detection in FTS3. file: [31f2ceee] check-in: [51525f9c] user: drh branch: trunk, size: 25064
21:40
Improved detection of corrupt shadow tables in FTS3. Enable the debugging special-inserts for FTS3 for both SQLITE_DEBUG and SQLITE_TEST. file: [b879664f] check-in: [04b2873b] user: drh branch: trunk, size: 24808
2019-01-25
18:17
Fix the assert_fts3_nc() macro so that it works for test builds that omit SQLITE_DEBUG. file: [74384e28] check-in: [b10df2b5] user: drh branch: trunk, size: 24777
14:48
Add an assert_fts3_nc() macro to fts3 - for assert() conditions that are only true when it is guaranteed that the fts3 database is not corrupt. file: [24ecc6a7] check-in: [3498908c] user: dan branch: trunk, size: 24747
2019-01-15
16:14
Fix a buffer overread in fts3 caused by a corrupt record. file: [6c666f31] check-in: [e54efd60] user: dan branch: trunk, size: 24366
2018-04-27
18:05
Update test script fts3expr4.test so that it always creates fts3 tokenizers in the "en_US" locality. file: [3378157f] check-in: [576a8f69] user: dan branch: trunk, size: 24335
2017-07-27
14:04
A demonstration of how FTS3 cursor pointers can be passed from the MATCH operator to the snippet() function securely and opaquely without having to use the sqlite3_result_pointer() and sqlite3_value_pointer() interfaces. file: [f4ee6393] check-in: [49cf31da] user: drh branch: pass-pointer-as-blob, size: 24468
2017-07-17
09:30
Use integer handles for fts3 cursors for a small performance improvement. file: [6e503a75] check-in: [f4f7196b] user: dan branch: fts3-int-cursor, size: 24664
2017-02-08
19:10
Avoid preparing a SELECT statement each time an UPDATE or DELETE by docid is executed against an fts3 table. file: [eb250200] check-in: [9962c10a] user: dan branch: fts3-seekstmt-cache, size: 24324
2016-02-11
22:41
Automatically disable FTS3 and FTS4 when building with SQLITE_OMIT_VIRTUALTABLE. file: [89d0bd45] check-in: [0beb32d2] user: drh branch: trunk, size: 24189
2015-09-28
15:23
Update fts3 to use the onepass strategy for delete operations. file: [c84125c6] check-in: [fffab4f7] user: dan branch: vtab-onepass, size: 24056
2015-05-25
10:57
Fix an fts4 matchinf() problem triggered by deferred tokens that are part of phrases that are part of AND expressions. file: [60174395] check-in: [f1e942a1] user: dan branch: trunk, size: 23980
2015-05-06
17:41
More optimization for the 'y' and 'b' matchinfo directives. file: [142837a7] check-in: [8c5b9fed] user: dan branch: fts3-matchinfo-y, size: 23919
2015-05-05
19:37
Optimizations for the matchinfo() function, particularly the 'y' flag. file: [bf61766e] check-in: [dddd7e18] user: dan branch: fts3-matchinfo-y, size: 23746
2015-05-01
14:07
Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3, and to ensure that error messages are not overwritten and thus leaked. file: [59ecaa2d] check-in: [605347e0] user: drh branch: trunk, size: 23796
2015-04-15
09:16
Remove an incorrect assert() statement from sqlite3Fts3Dequote(). file: [3626655d] check-in: [bd06eeb8] user: dan branch: trunk, size: 23748
2015-01-27
18:43
Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function. file: [394858c1] check-in: [245e8730] user: dan branch: trunk, size: 23573
2014-07-03
12:18
Change fts3/4 so that the "unicode61" is included in builds by default. It may now be excluded by defining SQLITE_DISABLE_FTS3_UNICODE. file: [53d4eca1] check-in: [0cc0230a] user: dan branch: trunk, size: 23439
2014-05-14
15:58
Fix various problems to do with segment promotion. Add test file fts4growth2.test, containing tests to check that the FTS index does not grow indefinitely as the table is updated. Allow the user to configure the number of segments merged simultaneously by the automerge option. file: [16cddf2d] check-in: [21491a9b] user: dan branch: fts4-experimental, size: 23439
2014-05-12
20:04
Experimental code to prevent FTS indexes from growing indefinitely as the table is updated. file: [b250b154] check-in: [b3b505a4] user: dan branch: fts4-experimental, size: 23427
2014-03-12
19:20
Changes to FTS to ensure that it does not access the database from within the xConnect method. file: [bdeb9015] check-in: [c67a52c3] user: dan branch: trunk, size: 23427
2013-11-12
17:46
When possible, have FTS use 32-bit operations to read varints from the database. file: [eb5f8029] check-in: [aa7ba302] user: dan branch: trunk, size: 23414
2013-10-02
08:04
Add a test to check that the new multi-token phrase optimization is actually helping. file: [8689f7cf] check-in: [bc3a2ed5] user: dan branch: fts4-docid-range-constraints, size: 23251
2013-09-30
11:42
Have fts4 full-text queries consider "docid<?" and similar constraints. file: [0634f768] check-in: [6622424a] user: dan branch: fts4-docid-range-constraints, size: 23066
2013-07-04
23:53
Modify several extensions to use the new exported function naming. Fix some shared library compilation issues. file: [c7a45166] check-in: [f2ab8747] user: mistachkin branch: extRefactor, size: 22476
2013-06-21
18:36
Remove an unused function declaration from the FTS3 source code. file: [a50cd231] check-in: [096ae1d8] user: drh branch: trunk, size: 22501
17:30
Add the "notindexed" option to fts4. file: [2659cdfc] check-in: [8ff2b8f5] user: dan branch: fts4-notindexed, size: 22539
2013-06-19
20:13
Add the languageid_bits= option to fts. Still some problems to work out. file: [9bef3710] check-in: [d36d7e68] user: dan branch: fts-languageid-bits, size: 22662
2013-06-11
14:22
Add the SQLITE_FTS3_MAX_EXPR_DEPTH compile time option. file: [cb4df04c] check-in: [24fc9d44] user: dan branch: trunk, size: 22469
2013-05-07
12:16
Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this. file: [0b167bed] check-in: [0ba67b64] user: dan branch: trunk, size: 22040
2013-04-29
18:07
Improve the error message issued when an FTS query exceeds the maximum allowable tree depth. file: [23ea0a2b] check-in: [f480b1fe] user: dan branch: trunk, size: 22028
2013-04-22
17:07
Update MSVC makefile for the fts3tokenize virtual table. Clear some harmless compiler warnings in the fts3tokenize code. file: [352c8a83] check-in: [2b3fed09] user: drh branch: trunk, size: 22019
2012-08-20
17:24
Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option. file: [1e588252] check-in: [e799222f] user: dan branch: trunk, size: 21957
2012-06-06
19:51
Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61. file: [81115435] check-in: [eccd6b65] user: dan branch: trunk, size: 21663
19:30
Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0". file: [11c71106] check-in: [790f76a5] user: dan branch: trunk, size: 21663
2012-05-25
17:50
Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators. file: [7b163fa2] check-in: [0c13570e] user: dan branch: fts4-unicode, size: 21579
2012-05-10
17:43
Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38]. file: [aca752b9] check-in: [4d3e1673] user: dan branch: trunk, size: 21434
2012-03-28
18:08
Do the accounting for incr-merge work in pages instead of blocks. file: [d03b8cf0] check-in: [83037d58] user: dan branch: fts4-incr-merge-exp, size: 21426
2012-03-27
15:00
Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables. file: [5fd2ec4e] check-in: [5c447e22] user: dan branch: fts4-incr-merge, size: 21427
2012-03-26
10:36
Add an experimental integrity-check function to FTS. file: [133e5c61] check-in: [40fc8804] user: dan branch: fts4-incr-merge, size: 21357
2012-03-24
17:29
Enable fts3 tables to use incremental merge by automatically creating the %_stat table when it is needed. file: [eb749124] check-in: [cc051fc0] user: drh branch: fts4-incr-merge, size: 21295
16:11
Remove the Fts3Table.mxLevel variable. file: [6d4ffaca] check-in: [67a0cffc] user: dan branch: fts4-incr-merge, size: 21244
14:45
Modify the way the number of leaves written and the maximum relative level are calculated in the auto-incr-merge code. file: [f65df0fd] check-in: [0d841c95] user: dan branch: fts4-auto-incr-merge, size: 21323
02:20
An attempt at automatic incremental merging for FTS4. file: [1e635c9b] check-in: [ed69434c] user: drh branch: fts4-auto-incr-merge, size: 21278
2012-03-23
18:26
Fix a spurious SQLITE_CONSTRAINT error that may be returned by an incr-merge operation. file: [5fe1651d] check-in: [ed7c17ea] user: dan branch: fts4-incr-merge, size: 21013
2012-03-17
16:56
Fix various incorrect and missing comments and other style issues in and around the FTS incremental merge code. file: [caa745f8] check-in: [7aabb62c] user: dan branch: fts4-incr-merge, size: 20935
2012-03-14
20:01
Add tests for incremental merge code. file: [1da6d2af] check-in: [57047372] user: dan branch: fts4-incr-merge, size: 20769
2012-03-08
18:39
Add the 'merge=?,?' command to fts4. This still needs some work. file: [cc8991da] check-in: [741b8f89] user: dan branch: fts4-incr-merge, size: 20769
2012-03-03
18:46
Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater. file: [d1d7f964] check-in: [f8e9c445] user: dan branch: fts4-languageid, size: 20714
2012-03-02
12:26
Fix the FTS 'optimize' command on multi-lingual databases. file: [521d300f] check-in: [65fa6937] user: dan branch: fts4-languageid, size: 20602
2012-03-01
19:44
Add the "languageid=" option to fts4. This code is still largely untested and alsmost certainly buggy. file: [8ba2d8ce] check-in: [bea257f7] user: dan branch: fts4-languageid, size: 20602
2012-01-25
16:29
Fix an FTS problem triggered by querying for an N character prefix using an N+1 character prefix index after rows have been deleted from the FTS table. Fix for [edb497982c]. file: [ce958a6f] check-in: [c05c3fd2] user: dan branch: trunk, size: 20346
2011-10-19
16:20
Merge the fts4-content branch with the trunk. file: [def7a900] check-in: [8a407705] user: dan branch: trunk, size: 20341
15:52
Have FTS3 ignore ^ prefixes. The ^ syntax is only supported on FTS4 tables. file: [74afec80] check-in: [df36ac94] user: dan branch: fts4-content, size: 20184
11:57
Change the way ^ tokens work in FTS so that the filtering is done as part of reading the FTS index instead of waiting until an entire doclist has been retrieved and then filtering it. file: [7a0deb21] check-in: [9b58c59e] user: dan branch: fts4-content, size: 20179
2011-10-18
19:39
If a token within an FTS query is prefixed with a '^' character, it must be the first token in a column of data to match. file: [bc27eebe] check-in: [63ac33c8] user: dan branch: fts4-content, size: 20070
2011-10-13
17:16
Handle updating the only row of an FTS table correctly. Fix for [9fd058691]. file: [59c5a947] check-in: [7e24645b] user: dan branch: trunk, size: 20046
2011-10-04
16:37
Add tests to check that modifying the schema of an FTS content table does not cause a crash in the FTS module. Also disable the deferred token optimization for content=xxx FTS tables. file: [06f442ce] check-in: [be86c706] user: dan branch: fts4-content, size: 19990
11:22
Add experimental 'content' option to FTS4. file: [a335d671] check-in: [1d27ea74] user: dan branch: fts4-content, size: 20021
2011-07-09
16:17
Fix harmless compiler warnings on unix. file: [30063fdd] check-in: [90b1aea1] user: drh branch: trunk, size: 19889
2011-07-07
07:37
Remove accidental global variable "doclist" from fts3Int.h. file: [d18022ed] check-in: [f244b2f3] user: dan branch: trunk, size: 20256
2011-06-28
09:51
Merge latest trunk changes with fts3-changes branch. file: [e670e2c5] check-in: [22668647] user: dan branch: fts3-changes, size: 20264
07:15
Changes to allow FTS to be compiled as a loadable module again. file: [61c90f51] check-in: [29e69f38] user: dan branch: trunk, size: 20365
2011-06-23
17:09
Fix some of the code issues (missing comments etc.) in the new FTS code. file: [7b452eeb] check-in: [8230d831] user: dan branch: fts3-changes, size: 20049
2011-06-20
19:00
More compiler warning fixes. file: [fa493ccb] check-in: [ed2dda93] user: drh branch: trunk, size: 20150
2011-06-17
17:37
Avoid loading doclists for infrequent terms that are part of phrases twice. file: [8ece4390] check-in: [8f939723] user: dan branch: trunk, size: 20150
16:04
Add a missing declaration to fts3Int.h. file: [974cf471] check-in: [3bfd4466] user: dan branch: trunk, size: 20202
2011-06-16
16:06
Fix a problem with NEAR queries executed inside a transaction that writes the FTS table. file: [df761492] check-in: [051c756c] user: dan branch: trunk, size: 20145
00:54
Changes to #ifdefs so that the build goes correctly if the only FTS macro defined is SQLITE_ENABLE_FTS4. file: [27b0b75c] check-in: [a0b43a32] user: drh branch: trunk, size: 20129
2011-06-08
18:39
Fix various issues to do with deferred tokens, NEAR expressions and matchinfo(). file: [a999cfbf] check-in: [3972a787] user: dan branch: fts3-prefix-search, size: 19758
2011-06-04
20:04
Allow the "order=DESC" and "order=ASC" parameters in FTS4 "CREATE VIRTUAL TABLE" statements. Tables created with "order=DESC" store all doclists in descending order, which allows optimizations normally applied to "ORDER BY docid ASC" queries to be used with "ORDER BY docid DESC" queries instead. file: [d76b021d] check-in: [f6a0193f] user: dan branch: fts3-prefix-search, size: 19775
2011-06-03
18:00
FTS changes: Remove unreachable code. Fix bugs. When processing a large doclist incrementally, read from disk incrementally too. file: [ba6f831f] check-in: [a4c7e282] user: dan branch: fts3-prefix-search, size: 19621
2011-06-02
19:57
Changes to improve performance and support LIMIT clauses on fts3 tables. This branch is unstable for now. file: [ab148907] check-in: [28149a78] user: dan branch: fts3-prefix-search, size: 19973
2011-05-28
15:57
Minor changes made while planning a larger change. file: [832f4d42] check-in: [84097a4c] user: dan branch: fts3-prefix-search, size: 17572
2011-05-25
18:34
Change fts4 so that the prefix= parameter is passes a comma-separated list of integers. For each integer N, a separate index of all prefixes of length N bytes is created. file: [deeeac21] check-in: [be59bf49] user: dan branch: fts3-prefix-search, size: 17813
2011-05-24
18:49
If the fts4 option prefix=1 is specified, have the fts4 module maintain an index of prefixes as well as terms. file: [02699211] check-in: [b5bdc639] user: dan branch: fts3-prefix-search, size: 17033
15:36
Do not invoke the xRollbackTo or xRelease methods of a virtual table without having first invoked an appropriate xSavepoint method. Add assert() statements to FTS3/4 to verify that this is happening in all cases. file: [c8c0011c] check-in: [651ef242] user: drh branch: trunk, size: 16114
2011-05-17
14:41
Avoid including fts3_term.c in the amalgamation, as it contains test code only. file: [05d14515] check-in: [f392b7ae] user: dan branch: trunk, size: 15227
2011-05-04
12:52
Optimize "ORDER BY rowid/docid DESC/ASC" clauses on FTS tables. file: [8c2ac39e] check-in: [13395121] user: dan branch: vtab-conflict, size: 15189
2011-02-02
17:30
Optimize handling of equality and range constraints on the "term" column of an fts4aux table. file: [945926ea] check-in: [386701de] user: dan branch: trunk, size: 15102
2011-02-01
16:34
Add virtual table module "fts4aux", used to inspect the full-text index of an fts4 table directly. Also add the "compress" and "uncompress" fts4 options. file: [d833c1df] check-in: [b010ddcc] user: dan branch: fts4aux, size: 15051
2010-11-26
16:49
Fix various compiler warnings. file: [a6c69c1c] check-in: [c412f612] user: drh branch: trunk, size: 13800
2010-11-24
11:51
Remove some unused code from fts3. Add tests to fts3matchinfo.test. file: [52c81862] check-in: [ae40b34c] user: dan branch: fts3-experimental, size: 13825
2010-11-23
19:16
Experimental changes to fts3 function matchinfo(). file: [0f7b9f18] check-in: [9cf0f2b7] user: dan branch: fts3-experimental, size: 13940
2010-11-02
17:41
Changes to allow FTS4 tables to be created without the underlying %_docsize table (in order to save space). file: [bbcd6d67] check-in: [31989b18] user: dan branch: trunk, size: 13572
2010-10-26
07:14
More coverage tests for fts3.c. file: [819bc266] check-in: [7a2f2864] user: dan branch: experimental, size: 13557
2010-10-23
19:07
Fixes for the matchinfo() function related to FTS4 common token handling. file: [11fa12ce] check-in: [deb80eac] user: dan branch: experimental, size: 13469
2010-10-22
16:44
Add missing comments and fix compiler warnings in new FTS3/4 code. Other minor fixes too. file: [068d8015] check-in: [1c9c70fe] user: dan branch: experimental, size: 12800
2010-10-21
15:49
Merge trunk changes into experimental branch. file: [f80be5ab] check-in: [fd1e5cad] user: dan branch: experimental, size: 12843
2010-10-20
18:56
Updates to FTS4 to improve performance and make more accurate cost estimates for prefix terms. file: [9fbe422f] check-in: [d0a450ce] user: dan branch: experimental, size: 12844
2010-10-19
14:08
Experimental changes to fts4 to try to selectively avoid loading very large doclists. file: [a640e4fb] check-in: [5ae0ba44] user: dan branch: experimental, size: 12721
2010-09-17
15:28
Have all FTS3 queries obtain a read or write table-lock at the shared-cache level before doing anything else. file: [b4f0b05c] check-in: [018e82c7] user: dan branch: trunk, size: 12332
2010-03-23
18:24
More commenting and documentation enhancements in FTS3. file: [70528ba8] check-in: [892e2867] user: drh branch: trunk, size: 12294
2010-02-03
19:55
Add a new full-text search variant that tracks the total number of documents and document sizes, to make ranking search results easier. Currently called FTS4. file: [df812ef3] check-in: [1b6e6094] user: drh branch: trunk, size: 11967
2010-01-06
17:19
Change the fts3 snippet function to return (hopefully) more relevant snippets in less time. file: [45bc7e28] check-in: [8a208223] user: dan branch: trunk, size: 11652
2010-01-02
19:02
Add experimental implementation of FTS3 functions matchinfo() and snippet() (not enabled by default). file: [9326800f] check-in: [51f7ee84] user: dan branch: trunk, size: 11697
2009-12-22
18:56
Add the experimental FTS3 matchinfo() function. Provides details of the match that may be used for result ranking and other purposes. file: [0d7c8d66] check-in: [37a1de02] user: dan branch: trunk, size: 11045
2009-12-12
09:51
Add coverage test cases for fts3. file: [6fdd41b4] check-in: [8fcb0478] user: dan branch: trunk, size: 10364
2009-12-10
16:04
Changes to fts3 to avoid flushing data to disk within a SELECT statement. file: [1419e297] check-in: [48c0db0e] user: dan branch: trunk, size: 10347
2009-12-08
19:05
Add tests to improve coverage of fts3. Associated bugfixes. file: [36f8e6d6] check-in: [f0eac417] user: dan branch: trunk, size: 10266
2009-12-03
17:36
Fix an incorrect assert() in fts3.c. Add further fts3 tests. file: [cc716c74] check-in: [75863c2d] user: dan branch: trunk, size: 10267
06:26
Updates to FTS3 to correct compiler warnings under MSVC. file: [515132f0] check-in: [37495b55] user: shaneh branch: trunk, size: 10201
2009-12-01
17:08
Use #include "sqlite3.h" instead of <sqlite3.h> in fts3Int.h. file: [7bc65d9a] check-in: [7737db49] user: dan branch: trunk, size: 9928
17:05
Add typedefs for i16 and u8 to fts3Int.h when not building via the amalgamation method. file: [2d5dab92] check-in: [fa56c1c4] user: dan branch: trunk, size: 9928
13:57
Changes to the TCL interface header to allow it to be compiled independently from the amalgamation. file: [58698ec4] check-in: [58113932] user: drh branch: trunk, size: 9778
2009-11-30
19:48
Test coverage improvements in the FTS3 porter stemmer. file: [cc08454b] check-in: [6d112bfd] user: drh branch: trunk, size: 8870
2009-11-20
02:24
Minor optimizations to fts3 code. file: [74b21db1] check-in: [b456eacb] user: dan branch: trunk, size: 8703
2009-11-18
15:35
Add some missing comments and fix some other issues in fts3 code. file: [f7488bbc] check-in: [2fe579e7] user: dan branch: fts3-refactor, size: 8318
2009-11-17
12:52
Improvements to the way fts3 reads the full-text index. file: [5c040c0f] check-in: [45c051e7] user: dan branch: fts3-refactor, size: 8327
2009-11-14
18:41
Further OOM testing for fts3 code. Add Tcl code implementing an integrity-check for fts3. file: [f8419da4] check-in: [c27d46b3] user: dan branch: fts3-refactor, size: 8111
2009-11-13
17:36
Start reworking fts3 code to match the rest of SQLite (code conventions, malloc-failure handling etc.). file: [a6aa1a4a] check-in: [30a92f11] user: dan branch: fts3-refactor, size: 8037 Added