/ File History
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

History of ext/fts3/fts3.c

2019-12-03
03:31
Avoid a harmless zero offset of a null pointer in FTS3, so as to not provoke unnecessary warnings from run-time checkers. file: [9a69143c] check-in: [85d95abe] user: drh branch: trunk, size: 204519
2019-11-26
02:03
Fix an assert() in fts3 that could fail due to corrupt database records. file: [3fd0b9c9] check-in: [5d9a3693] user: dan branch: trunk, size: 204441
2019-11-21
14:20
Prevent direct and indirect recursive content= options in fts3/4 and fts5. file: [f2e6743c] check-in: [2eb99732] user: dan branch: trunk, size: 204433
2019-11-20
13:31
Further improve detection of corrupt records in fts3. Also fix an error with check-in [dfcf081d842629a0] file: [9c2ac044] check-in: [a0f6d526] user: dan branch: trunk, size: 204020
2019-11-18
14:04
Improvements to detection of corruption in the %_stat shadow table of FTS4. file: [a4c64bf2] check-in: [6b67eba5] user: drh branch: trunk, size: 204020
11:14
Detect and prevent infinite recursion in fts3SelectLeaf() due to a malformed FTS3 btree. file: [48e9359e] check-in: [dfcf081d] user: drh branch: trunk, size: 203927
2019-11-17
02:41
Better detection of corruption in the %_stat and %_docsize shadow tables of FTS3. file: [493e7470] check-in: [1e449687] user: drh branch: trunk, size: 203771
2019-11-16
23:47
More improvements to shadow table corruption detection in FTS3. file: [ad760e49] check-in: [51525f9c] user: drh branch: trunk, size: 203134
21:40
Improved detection of corrupt shadow tables in FTS3. Enable the debugging special-inserts for FTS3 for both SQLITE_DEBUG and SQLITE_TEST. file: [6184d574] check-in: [04b2873b] user: drh branch: trunk, size: 203041
2019-11-01
13:37
Fix an fts3 assert() that could fail when operating on a database containing corrupt records. file: [c75aa32b] check-in: [7c52f547] user: dan branch: trunk, size: 203010
2019-09-28
11:19
In FTS3/4, the poslist end marker must be larger than any other possible poslist value, even on a corrupt poslist. file: [a01da95e] check-in: [752679ae] user: drh branch: trunk, size: 202958
2019-09-19
13:51
Fix an assert() in fts3 that could fail when accessing a corrupt database. file: [32e913e6] check-in: [601ce953] user: dan branch: trunk, size: 202909
2019-09-18
17:22
Add extra tests for the handling of corrupt records in fts3. file: [91a5b302] check-in: [40e29a47] user: dan branch: trunk, size: 202863
2019-03-08
13:27
Fix another segfault in fts3 that could occur with corrupt records. file: [33958a8c] check-in: [fcc24edd] user: dan branch: trunk, size: 202781
2019-01-28
16:50
Fix a buffer overread in fts3 that could occur when accessing a corrupt database. file: [5da1329c] check-in: [a9faf903] user: dan branch: trunk, size: 202767
2019-01-26
19:09
Fix a broken assert() in fts3. file: [014e2bef] check-in: [b8dd2d67] user: dan branch: rollback-abort, size: 202704
2019-01-25
14:48
Add an assert_fts3_nc() macro to fts3 - for assert() conditions that are only true when it is guaranteed that the fts3 database is not corrupt. file: [229d0d6e] check-in: [3498908c] user: dan branch: trunk, size: 202746
2019-01-24
17:41
Fix a buffer overread in fts3 that could occur in a prefix query on a corrupted database. file: [c8b68202] check-in: [d0d56893] user: dan branch: trunk, size: 202404
2019-01-21
17:57
Avoid deferencing a freed pointer following an OOM or SQLITE_CORRUPT error in the fts3 xDestroy method. file: [560cc692] check-in: [505ed9a4] user: dan branch: trunk, size: 202328
16:12
Remove a faulty assert() from fts3. file: [63602028] check-in: [6c33a303] user: dan branch: trunk, size: 202375
2019-01-18
21:17
Fix a memory leak introduced by [55c5d72a]. file: [90352639] check-in: [fbd681dc] user: dan branch: trunk, size: 202395
21:03
Fix an infinite loop caused by a corrupt database in fts3. Also an undefined left-shift in fts5. file: [ec300b0a] check-in: [55c5d72a] user: dan branch: trunk, size: 202326
2019-01-17
03:43
Ensure that the variable-length integer decoder in FTS3 never tries to left-shift a negative number. file: [230bfe65] check-in: [10ffc1fe] user: drh branch: trunk, size: 202146
2019-01-15
16:14
Fix a buffer overread in fts3 caused by a corrupt record. file: [9f441b5c] check-in: [e54efd60] user: dan branch: trunk, size: 202124
2019-01-12
00:12
Improved shadow table corruption detection in the matchinfo() function of FTS3. file: [51313386] check-in: [567be3bb] user: drh branch: trunk, size: 202108
2019-01-11
21:34
Fix a segfault in fts3 prompted by a corrupted database. file: [6b5029db] check-in: [2d7b1d1d] user: dan branch: trunk, size: 202060
2019-01-08
20:02
Use 64-bit math to compute the sizes of memory allocations in extensions. file: [4636bd25] check-in: [ca67f2ec] user: drh branch: trunk, size: 202051
2018-12-26
11:39
Avoid a left-shift of a negative value (undefined behaviour) when dealing with a corrupt database in fts3. Cherrypick of [b851d12474]. file: [5adba0be] check-in: [2fa63a8b] user: dan branch: branch-3.22, size: 201469
2018-12-22
09:39
Avoid a left-shift of a negative value (undefined behaviour) when dealing with a corrupt database in fts3. file: [6cf87a0f] check-in: [b851d124] user: dan branch: trunk, size: 201982
2018-12-19
17:05
Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. Cherrypick from commit [8201f4e1] on branch-3.18. file: [cac1ce08] check-in: [553a923c] user: dan branch: branch-3.19, size: 201188
16:03
Add interfaces sqlite3_bind_pointer(), sqlite3_result_pointer(), and sqlite3_value_pointer() used to safely move pointer values through SQL without exposing underlying memory address information. Cherrypick from commit [8201f4e1] on branch-3.18. file: [2e76a808] check-in: [4cb67252] user: dan branch: branch-3.9, size: 200914
14:44
Fix a crash that can follow an OOM in fts3 on this branch. file: [d890fb31] check-in: [de078148] user: dan branch: branch-3.19, size: 201360
01:57
Add extra defenses against strategically corrupt databases to fts3/4. file: [ba4d2e51] check-in: [882ef4e3] user: drh branch: branch-3.9, size: 201102
01:44
Add extra defenses against strategically corrupt databases to fts3/4. file: [db4466fa] check-in: [4f315e4a] user: drh branch: branch-3.19, size: 201303
01:38
Add extra defenses against strategically corrupt databases to fts3/4. file: [10aa75a0] check-in: [4bb21d82] user: drh branch: branch-3.18, size: 201754
01:30
Add extra defenses against strategically corrupt databases to fts3/4. file: [829e2943] check-in: [c255889b] user: drh branch: branch-3.22, size: 201419
2018-11-27
09:57
Add an "#ifndef SQLITE_DISABLE_FTS4_DEFERRED" block around function fts3EvalDeferredPhrase() in fts3 in order to silence a compiler warning. file: [65b8489e] check-in: [6f4b0d04] user: dan branch: trunk, size: 201932
2018-11-05
23:01
Initial code to make shadow tables read-only to ordinary SQL. The now xShadowName method is added to the sqlite3_module object and is used to identify potential shadow tables. The SQLITE_PREPARE_SHADOW argument to sqlite3_prepare_v3() is defined. It is designed to permit writing to shadow tables, but is currently an unused placeholder. file: [eddaba47] check-in: [31942b3d] user: drh branch: read-only-shadow, size: 201853
2018-11-03
16:51
Add extra defenses against strategically corrupt databases to fts3/4. file: [012384f0] check-in: [d44318f5] user: dan branch: trunk, size: 201427
2018-09-10
16:38
Fix a broken assert() in fts3.c. file: [53020b61] check-in: [564fa751] user: dan branch: trunk, size: 201395
2018-04-27
18:05
Update test script fts3expr4.test so that it always creates fts3 tokenizers in the "en_US" locality. file: [b577921c] check-in: [576a8f69] user: dan branch: trunk, size: 201394
2017-07-27
14:04
A demonstration of how FTS3 cursor pointers can be passed from the MATCH operator to the snippet() function securely and opaquely without having to use the sqlite3_result_pointer() and sqlite3_value_pointer() interfaces. file: [53413c37] check-in: [49cf31da] user: drh branch: pass-pointer-as-blob, size: 201554
03:48
Add a destructor argument to sqlite3_bind_pointer() and sqlite3_result_pointer(). file: [f1c58503] check-in: [3d9e841f] user: drh branch: pointer-with-destructor, size: 201387
2017-07-21
07:56
Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This changes is a cherry-pick of [f0f49224] via [69906880cee] and is ultimately inspired by check-in [72de49f2]. file: [8825caa4] check-in: [8201f4e1] user: drh branch: branch-3.18, size: 201722
07:45
Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This changes is a cherry-pick of [f0f49224] with version-specific edits. That check-in was inspired by check-in [72de49f2]. file: [c936a156] check-in: [69906880] user: drh branch: branch-3.9.2, size: 200931
04:08
Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This is a cherry-pick of [f0f49224] which was in turn inspired by [72de49f2]. file: [ca29618d] check-in: [5a3022e0] user: drh branch: branch-3.8.10, size: 199987
03:23
Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This is a cherry-pick of [f0f492245e95], which is in turn inspired by check-in [72de49f2]. file: [48fed98a] check-in: [a66a5b39] user: drh branch: branch-3.8.6, size: 197270
03:09
Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This changes is inspired by check-in [72de49f2] but is new implementation, not a cherry-pick. file: [eca471e5] check-in: [f0f49224] user: drh branch: branch-3.7.11, size: 176353
2017-07-17
09:30
Use integer handles for fts3 cursors for a small performance improvement. file: [839018ad] check-in: [f4f7196b] user: dan branch: fts3-int-cursor, size: 202040
00:40
Add an experimental "pointer type" parameter to sqlite3_bind_pointer(), sqlite3_result_pointer(), and sqlite3_value_pointer(). The pointer type is a string that must compare equal using strcmp() or else the pointer comes through as a NULL. file: [dfda8bb4] check-in: [211cce04] user: drh branch: pointer-types, size: 201384
2017-07-15
10:44
Add new interfaces sqlite3_bind_pointer(), sqlite3_result_pointer(), and sqlite3_value_pointer() used to safely move pointer values through SQL without exposing underlying memory address information. file: [127b3f96] check-in: [d6a44b35] user: drh branch: branch-3.17, size: 201505
2017-07-13
17:48
Update the FTS3 extension to use sqlite3_result_pointer() and sqlite3_value_pointer() for passing FTS3 cursor objects from the table into functions such as snippet(). file: [02fbd221] check-in: [0fa2e170] user: drh branch: bind-pointer, size: 201356
2017-07-05
18:48
Have fts3 virtual table cursors free internal resources when they reach EOF, instead of waiting until the xClose method is called. file: [b47d3ba5] check-in: [b6b14ab6] user: dan branch: trunk, size: 201464
2017-06-28
17:29
Instead of the new sqlite3_prepare_v3() interface, provide the SQLITE_DBCONFIG_PREPARE_FLAGS interface which sets the flags on the single next call to sqlite3_prepare_v2() or its cousins. file: [45ea4573] check-in: [942c3ef8] user: drh branch: dbconfig-prepare-flags, size: 201447
2017-06-01
00:54
Add interfaces sqlite3_prepare_v3() and sqlite3_prepare16_v3() with the extra prepFlags argument. Add the SQLITE_PREPARE_PERSISTENT option as one bit in that argument. file: [6c5543ce] check-in: [4a25c588] user: drh branch: prepare_v3, size: 201316
2017-05-02
18:00
Fix the fts3EvalAverageDocsize() routine so that it returns errors from sqlite3_reset() rather than always returning SQLITE_OK. file: [88ea5c44] check-in: [430f539c] user: drh branch: trunk, size: 201271
2017-04-19
13:25
Further improvements to coverage of fts3 module. file: [10fc2211] check-in: [6b21d0fd] user: dan branch: trunk, size: 201283
07:33
Further modifications and test cases to improve test coverage of fts3. file: [aba99577] check-in: [ea8a0d2c] user: dan branch: trunk, size: 201438
2017-04-18
13:50
Use sqlite3_table_column_metadata() instead of a SELECT statement to check for the existence of a %_stat table in fts3. This leads to smaller and easier to test code. file: [43dbd764] check-in: [dc2a4802] user: dan branch: trunk, size: 201455
05:49
Refactor the fts3ColumnMethod() function so that all branches can be covered. file: [1f7f2b23] check-in: [e47fdb49] user: dan branch: trunk, size: 201697
2017-04-17
16:07
Further improvements to test coverage in fts3. file: [1db0b708] check-in: [352413ee] user: dan branch: trunk, size: 201860
13:38
Improve coverage of code in fts3. file: [a05d3acd] check-in: [1a08a836] user: dan branch: trunk, size: 201791
2017-03-20
19:35
Avoid passing NULL as the second argument to memcpy() in fts3. file: [f0d5de1b] check-in: [49b93d97] user: dan branch: trunk, size: 201861
18:53
Fix some problems in fts3 found by address-sanitizer. file: [df964dbc] check-in: [16a8e84f] user: dan branch: trunk, size: 201838
2017-02-27
14:52
Add an sqlite3_set_last_insert_rowid() method. Use it to work around fts4 and fts5 modifying the last-insert-rowid unintuitively from within commit processing. file: [95c7041e] check-in: [fe41bb56] user: dan branch: set-last-insert-rowid, size: 201771
2017-02-08
19:10
Avoid preparing a SELECT statement each time an UPDATE or DELETE by docid is executed against an fts3 table. file: [c4d7eecb] check-in: [9962c10a] user: dan branch: fts3-seekstmt-cache, size: 201660
2017-02-01
22:43
Fix harmless compiler warnings seen with MSVC. file: [5a8cafed] check-in: [997f765b] user: mistachkin branch: msvcWarn, size: 201086
2015-09-29
11:57
Change the name of the new sqlite3_index_info.flags field to "idxFlags". Add documentation for the same to sqlite.h.in. file: [e028eb13] check-in: [f61203bc] user: dan branch: vtab-onepass, size: 201070
2015-09-28
15:23
Update fts3 to use the onepass strategy for delete operations. file: [52232f39] check-in: [fffab4f7] user: dan branch: vtab-onepass, size: 201067
2015-07-25
19:26
Remove a duplicated "if" condition from fts3.c. file: [b04b0c57] check-in: [ee348b12] user: dan branch: trunk, size: 200507
2015-07-08
17:59
Fix two problems that could cause fts3 auxiliary functions to occasionally misbehave if used with match expressions that contain both OR and NEAR. file: [d2f7981f] check-in: [372c1db2] user: dan branch: trunk, size: 200562
2015-05-25
10:57
Fix an fts4 matchinf() problem triggered by deferred tokens that are part of phrases that are part of AND expressions. file: [a95de519] check-in: [f1e942a1] user: dan branch: trunk, size: 200093
2015-05-21
02:24
Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer. Cherrypick [ad4b19d2ac0889a] file: [64d50e38] check-in: [000197cc] user: drh branch: branch-3.7.11, size: 176492
01:04
Do not allow virtual table constructors to be called recursively. Cherrypick [0a72726da21581ab] file: [a2a545a3] check-in: [0f0694e4] user: drh branch: branch-3.7.11, size: 176414
2015-05-20
20:34
Fix a problem causing the fts3 integrity-check to fail if run inside a transaction. Cherrypick of [3b925189a75e]. file: [47ca4807] check-in: [7d7d633c] user: dan branch: branch-3.8.6, size: 197409
19:50
Do not allow virtual table constructors to be called recursively. Cherrypick of [0a72726da215] file: [31686f9b] check-in: [023a29ba] user: dan branch: branch-3.8.6, size: 197401
19:32
Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer. Cherrypick of [ad4b19d2ac0889a2]. file: [d41ed0df] check-in: [ff7be4db] user: dan branch: branch-3.8.6, size: 197250
2015-05-05
19:37
Optimizations for the matchinfo() function, particularly the 'y' flag. file: [341e9d9a] check-in: [dddd7e18] user: dan branch: fts3-matchinfo-y, size: 200146
2015-05-01
14:07
Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3, and to ensure that error messages are not overwritten and thus leaked. file: [2fb98467] check-in: [605347e0] user: drh branch: trunk, size: 200126
2015-04-23
11:52
Fix a problem causing the fts3 integrity-check to fail if run inside a transaction. file: [81f9ed55] check-in: [3b925189] user: dan branch: trunk, size: 199896
2015-04-21
12:06
Fix an fts4 problem to do with the handling of phrases that contain zero tokens. file: [1b198ddb] check-in: [caba5be0] user: dan branch: trunk, size: 199888
2015-04-17
20:51
Fix a problem affecting some fts3 UPDATE and DELETE statements on order=DESC tables. file: [29300a76] check-in: [284c1623] user: dan branch: trunk, size: 199850
2015-04-16
20:10
Fix a boundry-value condition in the phrase poslist extractor of FTS3/4. file: [cb501902] check-in: [55d10baf] user: drh branch: trunk, size: 198926
2015-04-15
09:16
Remove an incorrect assert() statement from sqlite3Fts3Dequote(). file: [4bd75289] check-in: [bd06eeb8] user: dan branch: trunk, size: 198883
08:20
Fix a problem causing an assert() to fail if a snippet containing 0 tokens was requested from fts3. file: [ed6624d7] check-in: [eafd0a1e] user: dan branch: trunk, size: 198707
2015-04-10
07:55
Do not allow virtual table constructors to be called recursively. file: [57d863c3] check-in: [0a72726d] user: dan branch: trunk, size: 198624
2015-04-06
09:05
Fix a problem with fts3 prefix terms within phrase queries on "order=DESC" tables with a mix of negative and positive rowids. file: [23bd9d37] check-in: [0cdf5028] user: dan branch: fts3-prefix-query-fix, size: 198473
2015-03-26
14:29
Fix a harmless compiler warning in FTS3. file: [d3f6f0e9] check-in: [1688f60c] user: drh branch: trunk, size: 197195
2015-03-21
20:50
Add a missing "int" on a constant declaration. file: [2a1cf231] check-in: [235157de] user: drh branch: trunk, size: 197195
12:22
Avoid an integer overflow in fts3 causing gcc 4.7.1 with -O2 to behave counter-intuitively (perhaps because the behaviour is undefined). Add an "ifcapable trace" to a test in shell4.test. file: [e2c7e61d] check-in: [e3e23464] user: dan branch: trunk, size: 197191
2015-03-19
15:52
Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer. file: [022915f3] check-in: [ad4b19d2] user: dan branch: trunk, size: 197064
2015-01-29
11:52
Optimize range constraints on the rowid column of fts3/4 tables even if there is no MATCH clause in the query. file: [56a78f7e] check-in: [85dc1262] user: dan branch: trunk, size: 196986
2015-01-27
21:24
Fix harmless compiler warnings. file: [3b2f792a] check-in: [e7d2ec04] user: mistachkin branch: trunk, size: 196722
18:43
Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function. file: [845f2044] check-in: [245e8730] user: dan branch: trunk, size: 196803
2015-01-12
18:38
Fix some harmless compiler warnings. file: [5c464816] check-in: [2b8eecbf] user: mistachkin branch: msvcW4, size: 197236
2014-10-09
14:00
Fix a memory leak associated with the FTS4 matchinfo() function. file: [8b6cceb3] check-in: [fb8da824] user: drh branch: trunk, size: 197224
2014-10-06
12:41
Fix a harmless compiler warning inside an assert() in FTS4. file: [66f39c42] check-in: [418f3c9a] user: drh branch: trunk, size: 197190
2014-07-03
12:18
Change fts3/4 so that the "unicode61" is included in builds by default. It may now be excluded by defining SQLITE_DISABLE_FTS3_UNICODE. file: [2f5e925b] check-in: [0cc0230a] user: dan branch: trunk, size: 197172
2014-05-27
19:18
Fix harmless compiler warnings. file: [20bc6586] check-in: [20b75460] user: drh branch: trunk, size: 197172
2014-05-26
16:40
Fix a problem in FTS4 where columns with names that are prefixes of any notindexed column were also being (incorrectly) marked as not indexed. For example in "CREATE ... t1(abc, bc, abcd, notindexed=abcd)", both abc and abcd were being treated as notindexed. file: [0a9813c0] check-in: [d90c4964] user: dan branch: trunk, size: 197168
2014-05-14
15:58
Fix various problems to do with segment promotion. Add test file fts4growth2.test, containing tests to check that the FTS index does not grow indefinitely as the table is updated. Allow the user to configure the number of segments merged simultaneously by the automerge option. file: [e83f894c] check-in: [21491a9b] user: dan branch: fts4-experimental, size: 197134
2014-03-12
19:20
Changes to FTS to ensure that it does not access the database from within the xConnect method. file: [41b1920b] check-in: [c67a52c3] user: dan branch: trunk, size: 197084
2014-01-04
19:58
Add the usual "fts3" prefix to new static method setEstimatedRows() in fts3.c. This fixes a problem when compiling the amalgamation, as the r-tree module also contains a static method named setEstimatedRows. file: [3fe91e36] check-in: [d6fcfc88] user: dan branch: trunk, size: 196225
14:42
Instead of having the planner ignore plans with unusable MATCH constraints, have FTS assign extremely high costs to such plans in order to discourage the planner from using them. file: [2af2cb2e] check-in: [24f84b38] user: dan branch: avoid-unusable-match, size: 196217
2013-11-19
18:17
Fix a harmless MSVC compiler warning. file: [1e667eac] check-in: [6cc023bb] user: drh branch: trunk, size: 195222
2013-11-18
11:20
Fix harmless compiler warnings from clang scan-build. file: [3ae281c0] check-in: [8d002740] user: drh branch: trunk, size: 195215
2013-11-12
21:10
Fix harmless compiler warning. file: [dceaa507] check-in: [ddacd101] user: mistachkin branch: trunk, size: 195040
17:46
When possible, have FTS use 32-bit operations to read varints from the database. file: [7eb3b293] check-in: [aa7ba302] user: dan branch: trunk, size: 195051
2013-10-14
20:30
Fix a crash in FTS incremental phrase processing that can occur if the second or subsequent token is much more common in the dataset than the first. file: [f25ae572] check-in: [0bf438fc] user: dan branch: trunk, size: 194494
2013-10-12
02:33
Fix harmless compiler warning. file: [dcb90d12] check-in: [4b130f88] user: mistachkin branch: trunk, size: 194483
2013-10-11
15:05
Fix various harmless compiler warnings. Change the "warnings.sh" script to work with STAT4 instead of STAT3. file: [1864684d] check-in: [7df06684] user: drh branch: trunk, size: 194478
2013-10-07
21:49
Fix compilation issue with MSVC. file: [543cbd73] check-in: [36d64dc3] user: mistachkin branch: trunk, size: 194478
2013-10-03
19:27
Allow FTS4 multi-token phrases to use a combination of in-memory and incrementally loaded doclists. This allows phrases to (partially) benefit from incremental doclists without disabling the deferred token optimization. file: [6d277a3f] check-in: [f6819c5f] user: dan branch: fts4-docid-range-constraints, size: 194479
2013-10-02
08:04
Add a test to check that the new multi-token phrase optimization is actually helping. file: [a6b01535] check-in: [bc3a2ed5] user: dan branch: fts4-docid-range-constraints, size: 192860
2013-10-01
20:02
Allow multi-token phrases to load doclists from the database incrementally. This allows queries that feature such phrases to benefit from the "docid<?" optimization. file: [e781a014] check-in: [ea543f08] user: dan branch: fts4-docid-range-constraints, size: 192745
2013-09-30
18:16
Merge trunk changes with this branch. file: [00a56dc5] check-in: [e294a9c7] user: dan branch: fts4-docid-range-constraints, size: 188213
18:14
Fix a performance problem in the FTS4 auxiliary functions triggered by an OR clause in the full-text query. file: [e1240ab6] check-in: [fa0f2f0e] user: dan branch: trunk, size: 185441
11:42
Have fts4 full-text queries consider "docid<?" and similar constraints. file: [2dd0f461] check-in: [6622424a] user: dan branch: fts4-docid-range-constraints, size: 186342
2013-07-09
12:36
Modify several extensions to use the new exported function naming. Fix some shared library compilation issues. file: [0d6311cd] check-in: [1e39f850] user: drh branch: trunk, size: 183570
2013-07-05
10:46
Make a couple of implicit casts explicit to fix compiler warnings. file: [1370ea15] check-in: [9676280a] user: dan branch: trunk, size: 183532
2013-07-04
23:53
Modify several extensions to use the new exported function naming. Fix some shared library compilation issues. file: [cafcfaf8] check-in: [f2ab8747] user: mistachkin branch: extRefactor, size: 183558
2013-06-26
11:43
Cut over the next generation query planner. Increase the version number to 3.8.0. file: [6dbb5c42] check-in: [0fe31f60] user: drh branch: trunk, size: 183520
2013-06-21
17:30
Add the "notindexed" option to fts4. file: [3cb4afd0] check-in: [8ff2b8f5] user: dan branch: fts4-notindexed, size: 183519
2013-06-20
18:32
Fix some issues related to ORDER BY and fts tables with a non-zero languageid_bits setting. file: [e88a755a] check-in: [81527768] user: dan branch: fts-languageid-bits, size: 183194
16:22
Add extra tests for fts with a non-zero languageid_bits setting. Fix querying by docid with the same. file: [6b8ca2c8] check-in: [b1df00f3] user: dan branch: fts-languageid-bits, size: 183063
11:48
Add tests (and a fix) for large and small fts docid values with various languageid_bits settings. file: [b0c6f122] check-in: [8dc261b7] user: dan branch: fts-languageid-bits, size: 183032
2013-06-19
20:13
Add the languageid_bits= option to fts. Still some problems to work out. file: [2d86f9b3] check-in: [d36d7e68] user: dan branch: fts-languageid-bits, size: 183032
2013-06-04
13:07
Increase the estimated full-scan cost for an FTS3 table. file: [760fecd1] check-in: [8d9f29f8] user: drh branch: nextgen-query-plan-exp, size: 182255
2013-05-27
10:11
When writing to an FTS table, take an exclusive shared-cache lock on the %_segdir table before writing. Otherwise, an xCommit() call may report an SQLITE_LOCKED error. file: [931b3c83] check-in: [3cd2da42] user: dan branch: trunk, size: 182254
2013-05-07
12:16
Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this. file: [4bc160e6] check-in: [0ba67b64] user: dan branch: trunk, size: 182324
2013-04-29
18:07
Improve the error message issued when an FTS query exceeds the maximum allowable tree depth. file: [5c3d44d1] check-in: [f480b1fe] user: dan branch: trunk, size: 182307
2013-04-22
15:30
Add virtual table module 'fts3tokenize' to fts3. fts3tokenize provides SQL access to FTS tokenizer implementations. file: [784aadfb] check-in: [c5a4b21a] user: dan branch: trunk, size: 182411
2013-03-21
21:20
Many spelling fixes in comments. No changes to code. file: [0eaedfc6] check-in: [6f6e2d50] user: mistachkin branch: trunk, size: 182346
2012-10-25
19:25
Fix a bug in fts3 causing it to return SQLITE_NOMEM when NEAR, OR and some auxiliary functions were used together. Also a segfault in vtab.c that could follow an OOM condition. file: [a867cafa] check-in: [33f4f981] user: dan branch: trunk, size: 182344
2012-08-20
17:24
Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option. file: [ab90126e] check-in: [e799222f] user: dan branch: trunk, size: 182239
2012-06-06
19:51
Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61. file: [bb3107c0] check-in: [eccd6b65] user: dan branch: trunk, size: 182142
2012-05-26
18:28
If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer. file: [41824d0d] check-in: [e71495a8] user: dan branch: fts4-unicode, size: 182142
14:54
Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1. file: [aff512cb] check-in: [8f3e60aa] user: dan branch: fts4-unicode, size: 181968
2012-05-25
17:50
Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators. file: [81c77264] check-in: [0c13570e] user: dan branch: fts4-unicode, size: 181965
2012-05-10
17:43
Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38]. file: [a7adf674] check-in: [4d3e1673] user: dan branch: trunk, size: 181729
2012-05-04
02:58
Add an experimental tokenizer to FTS3/4: one that transliterates latin, greek, cyrillic, and hebrew characters into pure ascii. file: [5718d78a] check-in: [93011569] user: drh branch: translit-tokenizer, size: 178962
2012-03-30
00:00
Fix compiler warnings on GCC and MSVC and fix a C89-ism that broke the build for MSVC. file: [111626ce] check-in: [b451c0f9] user: drh branch: trunk, size: 178665
2012-03-27
15:00
Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables. file: [f41f52a2] check-in: [5c447e22] user: dan branch: fts4-incr-merge, size: 178660
00:38
Minor correct errors in the file format description for FTS3/4 contained in the fts3.c header comment. file: [5e2d74de] check-in: [fb8aacdd] user: drh branch: fts4-incr-merge, size: 178610
2012-03-26
14:36
Fix FTS3 so that it works even without SQLITE_DEBUG. file: [f2dd409e] check-in: [a18c1031] user: drh branch: fts4-incr-merge, size: 178556
10:36
Add an experimental integrity-check function to FTS. file: [a36f2add] check-in: [40fc8804] user: dan branch: fts4-incr-merge, size: 178568
2012-03-24
17:29
Enable fts3 tables to use incremental merge by automatically creating the %_stat table when it is needed. file: [95409b49] check-in: [cc051fc0] user: drh branch: fts4-incr-merge, size: 178655
16:43
Add a comment to fts3SyncMethod() to justify the nMinMerge=64 constant. file: [10988e19] check-in: [af55ca5f] user: dan branch: fts4-incr-merge, size: 178082
16:11
Remove the Fts3Table.mxLevel variable. file: [eb3a57bb] check-in: [67a0cffc] user: dan branch: fts4-incr-merge, size: 176929
14:45
Modify the way the number of leaves written and the maximum relative level are calculated in the auto-incr-merge code. file: [442f0ccf] check-in: [0d841c95] user: dan branch: fts4-auto-incr-merge, size: 176932
02:20
An attempt at automatic incremental merging for FTS4. file: [ef3d7af4] check-in: [ed69434c] user: drh branch: fts4-auto-incr-merge, size: 176597
2012-03-23
18:26
Fix a spurious SQLITE_CONSTRAINT error that may be returned by an incr-merge operation. file: [c47bb6fe] check-in: [ed7c17ea] user: dan branch: fts4-incr-merge, size: 176353
2012-03-19
14:57
Fix one more compiler warning missed by the previous check-in. file: [a62e0914] check-in: [bc03d99a] user: drh branch: trunk, size: 176263
14:51
Suppress harmless compiler warnings on windows in FTS4 and RTREE. file: [cb5b7760] check-in: [4fd68647] user: drh branch: trunk, size: 176258
14:28
Avoid a compiler warning (an incorrect compiler warning, at that) in vs2010. file: [bd94c662] check-in: [7dd97f12] user: drh branch: trunk, size: 176188
2012-03-05
15:33
Merge the fts4-languageid branch with the trunk. file: [806632fd] check-in: [99a9073b] user: dan branch: trunk, size: 176185
2012-03-03
18:46
Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater. file: [fcda9a9f] check-in: [f8e9c445] user: dan branch: fts4-languageid, size: 176349
2012-03-02
20:34
Remove an obsolete comment from FTS3. No code changes. file: [96ae9e27] check-in: [74d224b1] user: drh branch: trunk, size: 173516
19:53
Fix problems with combining content= and languageid= in a single fts4 table. file: [fd89caa4] check-in: [22491e7b] user: dan branch: fts4-languageid, size: 176335
2012-03-01
19:44
Add the "languageid=" option to fts4. This code is still largely untested and alsmost certainly buggy. file: [93a8eb6e] check-in: [bea257f7] user: dan branch: fts4-languageid, size: 175766
2012-01-25
16:29
Fix an FTS problem triggered by querying for an N character prefix using an N+1 character prefix index after rows have been deleted from the FTS table. Fix for [edb497982c]. file: [4cf7b8e5] check-in: [c05c3fd2] user: dan branch: trunk, size: 173680
2011-11-04
10:37
Add a missing va_end() macro to fts3.c. file: [bd570b99] check-in: [e6f82574] user: dan branch: trunk, size: 173631
2011-10-19
18:21
Fix harmless compiler warnings in FTS4. file: [246ef2d0] check-in: [1af4a256] user: drh branch: trunk, size: 173615
16:20
Merge the fts4-content branch with the trunk. file: [9c125699] check-in: [8a407705] user: dan branch: trunk, size: 173577
15:52
Have FTS3 ignore ^ prefixes. The ^ syntax is only supported on FTS4 tables. file: [e8ee5c78] check-in: [df36ac94] user: dan branch: fts4-content, size: 173359
11:57
Change the way ^ tokens work in FTS so that the filtering is done as part of reading the FTS index instead of waiting until an entire doclist has been retrieved and then filtering it. file: [064b660a] check-in: [9b58c59e] user: dan branch: fts4-content, size: 173346
09:40
Fix a problem in FTS to do with ^ tokens and the snippet() function. file: [708122f0] check-in: [2c03b24f] user: dan branch: fts4-content, size: 173576
2011-10-18
19:39
If a token within an FTS query is prefixed with a '^' character, it must be the first token in a column of data to match. file: [e12a151b] check-in: [63ac33c8] user: dan branch: fts4-content, size: 173552
12:49
Cherrypick patch [3126754c72] from the trunk into the content= branch. file: [15e1725f] check-in: [f9b5b217] user: dan branch: fts4-content, size: 171753
12:44
Fix a floating-point exception that can occur when an FTS4 query contains a large number of tokens connected by AND or NEAR operators. file: [19f36945] check-in: [3126754c] user: dan branch: trunk, size: 165665
2011-10-15
00:16
Added the tool/warnings-clang.sh script. Changes so that there are no warnings with either gcc or clang even including FTS4 and RTREE and both with and without SQLITE_THREADSAFE=0. file: [d02bb1b3] check-in: [39408702] user: drh branch: trunk, size: 165444
2011-10-14
22:57
Fix a few minor and harmless clang warnings in FTS3 and RTREE. file: [9dd7e3ed] check-in: [b3324f6c] user: drh branch: trunk, size: 165427
2011-10-13
17:16
Handle updating the only row of an FTS table correctly. Fix for [9fd058691]. file: [0ace6b45] check-in: [7e24645b] user: dan branch: trunk, size: 165337
2011-10-05
15:11
Change FTS4 so that if both the content=xxx option and column names are specified, the virtual table assumes that the named columns correspond to columns of table xxx. file: [f2ed0ae6] check-in: [289ee431] user: dan branch: fts4-content, size: 171532
2011-10-04
19:41
Improve test coverage of fts3.c. file: [6d6f3d33] check-in: [0f439944] user: dan branch: fts4-content, size: 170826
16:37
Add tests to check that modifying the schema of an FTS content table does not cause a crash in the FTS module. Also disable the deferred token optimization for content=xxx FTS tables. file: [df149056] check-in: [be86c706] user: dan branch: fts4-content, size: 170553
11:22
Add experimental 'content' option to FTS4. file: [626fcf47] check-in: [1d27ea74] user: dan branch: fts4-content, size: 170158
2011-09-13
19:08
Allocate the correct size for the output buffer in fts3DoclistOrMerge(). Fix for [56be976859]. file: [195e4da6] check-in: [07788c0f] user: dan branch: trunk, size: 165226
2011-06-28
11:58
Add a fix and tests for the FTS deferred token logic. file: [f45ad450] check-in: [91daea7d] user: dan branch: fts3-changes, size: 163543
2011-06-27
11:15
Changes to improve the selection of deferred tokens within phrases. file: [70127bf5] check-in: [2c4bbd90] user: dan branch: fts3-changes, size: 162725
2011-06-23
17:09
Fix some of the code issues (missing comments etc.) in the new FTS code. file: [c25723c7] check-in: [8230d831] user: dan branch: fts3-changes, size: 162566
00:59
If "PRAGMA page_size" commands are not authorized, the FTS module will assume a page size of 1024. file: [ca776037] check-in: [ba39382e] user: drh branch: trunk, size: 143667
2011-06-20
20:50
Fix a couple of compiler issues on Windows due to warning clean up. file: [1c8a45df] check-in: [81e6f151] user: shaneh branch: trunk, size: 143583
19:00
More compiler warning fixes. file: [f63086fc] check-in: [ed2dda93] user: drh branch: trunk, size: 143583
17:24
Fix various harmless compiler warnings in FTS3 and RTREE. file: [e9c4b571] check-in: [fe62179e] user: drh branch: trunk, size: 143573
2011-06-17
17:37
Avoid loading doclists for infrequent terms that are part of phrases twice. file: [f919a796] check-in: [8f939723] user: dan branch: trunk, size: 143569
2011-06-16
16:06
Fix a problem with NEAR queries executed inside a transaction that writes the FTS table. file: [78b02b5f] check-in: [051c756c] user: dan branch: trunk, size: 142882
00:54
Changes to #ifdefs so that the build goes correctly if the only FTS macro defined is SQLITE_ENABLE_FTS4. file: [014e06f5] check-in: [a0b43a32] user: drh branch: trunk, size: 142856
2011-06-15
12:43
Fix a couple of compiler warnings. file: [6c7b5887] check-in: [3899f3b9] user: drh branch: broken-build, size: 142857
2011-06-14
09:00
Fix another bug caused by NEAR/matchinfo/order=DESC interaction. file: [ae3ae91d] check-in: [04907fba] user: dan branch: fts3-prefix-search, size: 142858
07:22
Merge recent trunk changes into fts3-prefix-search branch. file: [5d59c015] check-in: [135ce30f] user: dan branch: fts3-prefix-search, size: 142910
07:14
Remove unused parameters from internal fts3 function. file: [f4f65273] check-in: [06de3f2c] user: dan branch: fts3-prefix-search, size: 142518
2011-06-13
17:00
Add tests for deferred tokens to fts3auto.test. Fix a problem with OR queries and deferred tokens. file: [2a48945d] check-in: [b9fb69e5] user: dan branch: fts3-prefix-search, size: 142008
09:11
Fix a bug exposed by combining matchinfo(), NEAR and "ORDER BY rowid DESC". file: [e71dafb1] check-in: [5f6b87f4] user: dan branch: fts3-prefix-search, size: 141836
2011-06-09
10:48
Fix problems to do with using both OR and NEAR operators in a single expression. file: [5df3b579] check-in: [4e8dd19e] user: dan branch: fts3-prefix-search, size: 141755
2011-06-08
18:39
Fix various issues to do with deferred tokens, NEAR expressions and matchinfo(). file: [b44083ca] check-in: [3972a787] user: dan branch: fts3-prefix-search, size: 141145
2011-06-07
18:35
Have NEAR queries use incremental merging. Fix issues surrounding the deferred token optimization. file: [9d2d2cab] check-in: [9d10a684] user: dan branch: fts3-prefix-search, size: 143648
2011-06-06
18:14
Merge the latest trunk changes into the fts3-prefix-search branch. file: [d0970d9f] check-in: [567dd843] user: drh branch: fts3-prefix-search, size: 137056
14:51
Modify fts3rnd.test to run tests for both "ORDER BY docid ASC" and "ORDER BY docid DESC" with both order=ASC and order=DESC FTS tables. Fixes for some bugs found. file: [aafc143b] check-in: [89f2f482] user: dan branch: fts3-prefix-search, size: 136933
06:55
Clean up the code for processing FTS4 options a bit. file: [ea03065c] check-in: [0425138a] user: dan branch: fts3-prefix-search, size: 136339
2011-06-04
20:13
Remove some unreachable code. file: [31f73cae] check-in: [650e1a79] user: dan branch: fts3-prefix-search, size: 135716
20:04
Allow the "order=DESC" and "order=ASC" parameters in FTS4 "CREATE VIRTUAL TABLE" statements. Tables created with "order=DESC" store all doclists in descending order, which allows optimizations normally applied to "ORDER BY docid ASC" queries to be used with "ORDER BY docid DESC" queries instead. file: [432c902c] check-in: [f6a0193f] user: dan branch: fts3-prefix-search, size: 142120
2011-06-03
18:00
FTS changes: Remove unreachable code. Fix bugs. When processing a large doclist incrementally, read from disk incrementally too. file: [3183cf6a] check-in: [a4c7e282] user: dan branch: fts3-prefix-search, size: 135253
2011-06-02
19:57
Changes to improve performance and support LIMIT clauses on fts3 tables. This branch is unstable for now. file: [f92b6e02] check-in: [28149a78] user: dan branch: fts3-prefix-search, size: 156042
2011-05-28
15:57
Minor changes made while planning a larger change. file: [eb59cdd8] check-in: [84097a4c] user: dan branch: fts3-prefix-search, size: 133250
2011-05-25
23:18
Fix some minor and harmless compiler warnings. file: [b3a10a1a] check-in: [a4755e70] user: drh branch: trunk, size: 128741
19:17
If a prefix index of size N is not present, use a prefix index of size N+1 along with the terms index for queries for prefixes of length N. file: [b44e0abc] check-in: [cc83991c] user: dan branch: fts3-prefix-search, size: 132878
18:47
Merge trunk changes into experimental fts3-prefix-search branch. file: [925360f6] check-in: [f0f0a03d] user: dan branch: fts3-prefix-search, size: 131731
18:34
Change fts4 so that the prefix= parameter is passes a comma-separated list of integers. For each integer N, a separate index of all prefixes of length N bytes is created. file: [9ebda4cd] check-in: [be59bf49] user: dan branch: fts3-prefix-search, size: 131750
01:16
Changes to savepoint in virtual tables for simpler and more consistent operation. file: [0a2ae110] check-in: [92f26a8b] user: drh branch: trunk, size: 128618
2011-05-24
18:49
If the fts4 option prefix=1 is specified, have the fts4 module maintain an index of prefixes as well as terms. file: [4a48bfef] check-in: [b5bdc639] user: dan branch: fts3-prefix-search, size: 129763
15:36
Do not invoke the xRollbackTo or xRelease methods of a virtual table without having first invoked an appropriate xSavepoint method. Add assert() statements to FTS3/4 to verify that this is happening in all cases. file: [e7600e28] check-in: [651ef242] user: drh branch: trunk, size: 128637
2011-05-17
15:56
Add extended return code SQLITE_CORRUPT_VTAB. Returned when the tcontents of the sqlite tables used internally by a virtual table module are invalid or inconsistent. file: [18c2a079] check-in: [8844e8bf] user: dan branch: trunk, size: 127722
2011-05-06
00:19
Increase the iVersion for sqlite3_module needed for the xSavepoint, xRelease, and xRollbackTo methods from 1 to 2. This is so that it will conform to published documentation. file: [0077bd07] check-in: [0f4954e9] user: drh branch: trunk, size: 127707
2011-05-04
17:23
Merge vtab-conflict branch with trunk. file: [73d6718d] check-in: [8f9666af] user: dan branch: trunk, size: 127707
16:30
Fix a couple of compiler warnings in the FTS code. file: [84dc693d] check-in: [1a113359] user: dan branch: vtab-conflict, size: 127708
15:41
Fix a performance problem in queries that use "ORDER BY rowid DESC" and one or more FTS auxiliary functions. file: [18bf7e4e] check-in: [95e09b20] user: dan branch: vtab-conflict, size: 127323
12:52
Optimize "ORDER BY rowid/docid DESC/ASC" clauses on FTS tables. file: [47e4f4da] check-in: [13395121] user: dan branch: vtab-conflict, size: 126240
2011-04-26
19:21
Extra tests for fts3. And fixes for conflict-handling related problems in fts3. file: [ce37973c] check-in: [fb4a3558] user: dan branch: vtab-conflict, size: 124536
2011-04-25
18:49
Add support for on conflict clauses to fts3/fts4. file: [94fa15fc] check-in: [6d2633a6] user: dan branch: vtab-conflict, size: 123952
2011-02-03
12:48
Extra tests for fts4 compress/uncompress hooks. Fix some minor problems with the same. file: [5653c565] check-in: [80225abe] user: dan branch: trunk, size: 123884
2011-02-02
17:30
Optimize handling of equality and range constraints on the "term" column of an fts4aux table. file: [f93f5d61] check-in: [386701de] user: dan branch: trunk, size: 122882
2011-02-01
16:34
Add virtual table module "fts4aux", used to inspect the full-text index of an fts4 table directly. Also add the "compress" and "uncompress" fts4 options. file: [871600c0] check-in: [b010ddcc] user: dan branch: fts4aux, size: 122683
2010-12-10
17:06
Fix a performance regression in matchinfo(). file: [28ada7d1] check-in: [fa37d8eb] user: dan branch: trunk, size: 118524
2010-12-04
18:20
Fix a memory leak in fts3 that can occur in an OOM condition. file: [44f99865] check-in: [507027b7] user: dan branch: trunk, size: 118328
2010-12-02
17:39
Fix a bug causing fts3 to incorrectly return SQLITE_CORRUPT. file: [bb233da0] check-in: [099195b1] user: dan branch: trunk, size: 118179
2010-12-01
15:36
Fix some warnings under MSVC in fts3 module. file: [c354e8a4] check-in: [c7771c0b] user: shaneh branch: trunk, size: 118180
2010-11-26
16:49
Fix various compiler warnings. file: [bae65cf7] check-in: [c412f612] user: drh branch: trunk, size: 118152
16:31
Fix an uninitialized variable in fts3.c. file: [7fda91bb] check-in: [3c3d076b] user: dan branch: trunk, size: 118228
2010-11-24
15:02
Fix crashes that can occur when queries are run on an FTS4 table containing zero rows. file: [0e207ae8] check-in: [ed61fd20] user: dan branch: fts3-experimental, size: 118228
11:51
Remove some unused code from fts3. Add tests to fts3matchinfo.test. file: [916091c7] check-in: [ae40b34c] user: dan branch: fts3-experimental, size: 118130
2010-11-23
19:16
Experimental changes to fts3 function matchinfo(). file: [591b6fdb] check-in: [9cf0f2b7] user: dan branch: fts3-experimental, size: 118091
2010-11-18
13:52
Restrict the scope of the fts3ExprCost() subroutine inside of FTS3. file: [7e01fd82] check-in: [76681870] user: drh branch: trunk, size: 117936
13:47
Fix compiler warnings. file: [98e6f7c0] check-in: [6c4f1d5c] user: drh branch: trunk, size: 117929
2010-11-02
17:41
Changes to allow FTS4 tables to be created without the underlying %_docsize table (in order to save space). file: [9da9f1d5] check-in: [31989b18] user: dan branch: trunk, size: 118942
2010-10-30
15:21
Test cases and minor changes to make fts3 more robust in the face of a corrupt database. file: [48c37f64] check-in: [b7702905] user: dan branch: trunk, size: 116744
2010-10-26
07:14
More coverage tests for fts3.c. file: [e7827390] check-in: [7a2f2864] user: dan branch: experimental, size: 115965
2010-10-25
19:01
Further improvements to coverage of fts3.c. Fixes for bugs revealed by the same. file: [989ab870] check-in: [918b6092] user: dan branch: experimental, size: 115995
12:47
Test coverage improvements for fts3.c. file: [8a3b4144] check-in: [a8b1d998] user: dan branch: experimental, size: 115917
2010-10-23
19:07
Fixes for the matchinfo() function related to FTS4 common token handling. file: [b7442369] check-in: [deb80eac] user: dan branch: experimental, size: 116388
2010-10-22
16:44
Add missing comments and fix compiler warnings in new FTS3/4 code. Other minor fixes too. file: [e2f031ea] check-in: [1c9c70fe] user: dan branch: experimental, size: 115406
2010-10-21
15:49
Merge trunk changes into experimental branch. file: [f423181b] check-in: [fd1e5cad] user: dan branch: experimental, size: 109410
2010-10-20
18:56
Updates to FTS4 to improve performance and make more accurate cost estimates for prefix terms. file: [ce7bcd1f] check-in: [d0a450ce] user: dan branch: experimental, size: 109382
2010-10-19
14:08
Experimental changes to fts4 to try to selectively avoid loading very large doclists. file: [9d4ccf3b] check-in: [5ae0ba44] user: dan branch: experimental, size: 107042
2010-09-17
15:28
Have all FTS3 queries obtain a read or write table-lock at the shared-cache level before doing anything else. file: [03be86c5] check-in: [018e82c7] user: dan branch: trunk, size: 92319
2010-08-24
16:59
Fixes to allow fts3 tables to be renamed mid-transaction. file: [f6436b3b] check-in: [d1c87532] user: dan branch: trunk, size: 92250
2010-08-09
14:26
Fix two comment typos that were reported on the mailing list. file: [e818310c] check-in: [88bf9016] user: drh branch: trunk, size: 92200
2010-07-19
11:16
Re-introduce the prefix-search optimization of [feef1b15d6], which was lost in a reorganization of FTS3 code. file: [9dec342f] check-in: [d692434b] user: dan branch: trunk, size: 92199
2010-07-02
18:15
Fix a couple more compiler warnings under MSVC. file: [51948505] check-in: [26bc27e3] user: shaneh branch: mistake, size: 90068
2010-03-23
18:24
More commenting and documentation enhancements in FTS3. file: [20c55bed] check-in: [892e2867] user: drh branch: trunk, size: 89989
15:46
Fix up obsolete comments in FTS3 to conform to the latest nomenclature. Add new comments to better explain FTS3 operation. file: [e3b543c6] check-in: [3e4a0082] user: drh branch: trunk, size: 90107
2010-03-18
16:34
Mark internal routines as "static" in FTS3. Ticket [b1552d0fa7]. file: [2d2ee8cb] check-in: [39b34ac4] user: drh branch: trunk, size: 86266
2010-03-15
20:53
Improved error message when the right-hand operand of MATCH in FTS3 is not a valid search expression. Ticket [170872f1c3]. file: [2bb2045d] check-in: [bea92586] user: drh branch: trunk, size: 86259
19:27
Fix FTS3 so that it does not run illegal SQL and cause a spurious log message. Ticket [42d45a693e6]. file: [54d3c227] check-in: [f186b6a6] user: drh branch: trunk, size: 86079
2010-03-09
12:35
Disable FTS4 (which is incomplete and not ready for publication) to prevent accidental use. file: [dea6740e] check-in: [87a6a997] user: drh branch: trunk, size: 85869
2010-03-05
17:27
Fix a one byte buffer overread in fts3. file: [15ca4d9b] check-in: [6e3e014a] user: dan branch: trunk, size: 85762
2010-02-03
19:55
Add a new full-text search variant that tracks the total number of documents and document sizes, to make ranking search results easier. Currently called FTS4. file: [c7a20cf4] check-in: [1b6e6094] user: drh branch: trunk, size: 85460
2010-01-11
12:00
Modify snippets code to run more efficiently. And to avoid a bug relating to snippets based on full-text queries that contain duplicate terms. file: [699abb6c] check-in: [a2b1183d] user: dan branch: trunk, size: 83349
2010-01-08
23:01
Update comments in fts3.c to more accurately describe the doclist format. file: [451eb655] check-in: [e424a030] user: drh branch: trunk, size: 83342
2010-01-07
10:54
Fixes to problems in FTS3 snippet() function found by th3 tests. file: [7b1969f6] check-in: [3b5ccd26] user: dan branch: trunk, size: 82277
2010-01-06
17:19
Change the fts3 snippet function to return (hopefully) more relevant snippets in less time. file: [04e95afa] check-in: [8a208223] user: dan branch: trunk, size: 82220
2010-01-02
19:02
Add experimental implementation of FTS3 functions matchinfo() and snippet() (not enabled by default). file: [15fb87c1] check-in: [51f7ee84] user: dan branch: trunk, size: 81828
2009-12-31
19:06
Changes to remove warnings in MSVC build. file: [ac757a45] check-in: [6cf76c2a] user: shaneh branch: trunk, size: 86624
2009-12-29
10:32
Fix a bug triggered by optimizing an FTS3 table when there are no segments on disk but pending terms in the hash table. file: [439bda67] check-in: [a3b50e4f] user: dan branch: trunk, size: 86594
2009-12-22
18:56
Add the experimental FTS3 matchinfo() function. Provides details of the match that may be used for result ranking and other purposes. file: [0a09deb5] check-in: [37a1de02] user: dan branch: trunk, size: 86236
2009-12-20
15:00
Fix a problem in FTS3 phrase queries. Add tests to check that it really is fixed. file: [9c4ab6e7] check-in: [956de051] user: dan branch: trunk, size: 76803
2009-12-14
15:17
Handle an unhandled OOM error code that occurs in fts3 with SQLITE_TEST defined. file: [682882c1] check-in: [82b06999] user: dan branch: trunk, size: 76750
14:49
Fix a broken assert() in fts3_write.c. Also fix a couple of compiler warnings in fts3 code. file: [6ae94945] check-in: [a730a05b] user: dan branch: trunk, size: 76716
2009-12-12
19:15
Tests to cover a few extra branches in fts3.c. file: [ea601b21] check-in: [06b72b00] user: dan branch: trunk, size: 76726
09:51
Add coverage test cases for fts3. file: [e1828210] check-in: [8fcb0478] user: dan branch: trunk, size: 76739
2009-12-11
16:03
Change the fts3 test interface used to configure the advisory node size parameter. file: [c059f8e7] check-in: [87fc0ce1] user: dan branch: trunk, size: 76626
2009-12-10
18:38
Remove some unused macros and types from fts3.c. file: [b79e729c] check-in: [83a80efe] user: dan branch: trunk, size: 76968
18:29
Fix handling of "WHERE col MATCH ? AND docid = ?" clauses in fts3. file: [2542d222] check-in: [6cbbae84] user: dan branch: trunk, size: 78588
18:20
Fix an OOM related problem in the snippet() and offsets() functions of fts3. file: [a52050eb] check-in: [61efff41] user: dan branch: trunk, size: 78485
16:04
Changes to fts3 to avoid flushing data to disk within a SELECT statement. file: [f72d7fdb] check-in: [48c0db0e] user: dan branch: trunk, size: 78148
2009-12-09
14:39
Improve test coverage of fts3.c. file: [6e24afa9] check-in: [56b6432f] user: dan branch: trunk, size: 77849
05:30
Mark fts3ReallocOrFree and fts3InitVtab as static. Ticket [ff44d82f3b]. file: [cb9bd52e] check-in: [a9038306] user: dan branch: trunk, size: 77729
2009-12-08
19:05
Add tests to improve coverage of fts3. Associated bugfixes. file: [ece6d5b1] check-in: [f0eac417] user: dan branch: trunk, size: 77722
2009-12-07
16:18
Move some SQLITE_TEST code down to avoid mixing code and variable declarations. Fix a test for high-order bit handling in sqlite3Fts3InitTokenizer(). file: [684a55d6] check-in: [fad43d29] user: shaneh branch: trunk, size: 77665
12:34
Add some tests for OR, AND and NOT operations to fts3rnd.test. Add tests to check that errors are returned when bad arguments are passed to fts3 functions snippet, offsets and optimize. Minor fix for the same file: [43d8cfd0] check-in: [5811df3f] user: dan branch: trunk, size: 77665
2009-12-05
14:29
Fix another bug in 3-way NEAR queries. file: [f4152f27] check-in: [3bb13a06] user: dan branch: trunk, size: 77306
11:37
Fix some problems with FTS3 and 3-way NEAR queries. file: [7d344410] check-in: [23dc6fb5] user: dan branch: trunk, size: 77103
2009-12-04
19:07
Fix a problem involving a 3-way NEAR query. file: [54ad3e8f] check-in: [507890a9] user: dan branch: trunk, size: 75216
14:11
Modify [2ad1461f25] to avoid leaving a prepared statement in "active" state following an OOM error in FTS3. file: [da95f019] check-in: [69c21ee4] user: dan branch: trunk, size: 74883
13:43
Fix an FTS3 problem where an OOM error was not being propagated back out to the top-level interface. file: [fb2459bc] check-in: [2ad1461f] user: drh branch: trunk, size: 74397
2009-12-03
17:36
Fix an incorrect assert() in fts3.c. Add further fts3 tests. file: [b15d44a4] check-in: [75863c2d] user: dan branch: trunk, size: 74357
06:26
Updates to FTS3 to correct compiler warnings under MSVC. file: [8352dc35] check-in: [37495b55] user: shaneh branch: trunk, size: 73485
2009-12-01
13:57
Changes to the TCL interface header to allow it to be compiled independently from the amalgamation. file: [eca3828c] check-in: [58113932] user: drh branch: trunk, size: 73087
2009-11-28
17:23
Change FTS3 to detect when the RHS of the MATCH opertor encounters an OOM during string format conversion and report back an SQLITE_NOMEM error. file: [8e966349] check-in: [31eed4f8] user: drh branch: trunk, size: 73371
15:35
Add a test case for creating an FTS3 table with no module arguments or opening/closing brackets in the CREATE VIRTUAL TABLE statement. file: [591a8493] check-in: [a9cba7ea] user: dan branch: trunk, size: 73272
12:40
Fix a bug introduced by the fts3 refactoring (segfault when creating a table with zero module args). Also a fix to handle an OOM error. file: [524b080a] check-in: [eada284b] user: dan branch: trunk, size: 73165
2009-11-27
12:14
Remove a C++ism accidentally added to fts3.c. file: [c2776bb3] check-in: [97d33241] user: dan branch: trunk, size: 72918
2009-11-21
03:03
Add some missing comments to fts3 files. No source code changes. file: [6396a59d] check-in: [b6402b20] user: dan branch: trunk, size: 72902
2009-11-20
02:24
Minor optimizations to fts3 code. file: [4990e02c] check-in: [b456eacb] user: dan branch: trunk, size: 72190
2009-11-19
18:28
Fix a performance regression introduced while reworking the fts3 code. file: [e7383c74] check-in: [7cd178a7] user: dan branch: trunk, size: 71778
00:15
Fix problems introduced into fts3 as part of the refactoring. file: [3251d3c5] check-in: [fa0998e1] user: dan branch: fts3-refactor, size: 71670
2009-11-18
15:35
Add some missing comments and fix some other issues in fts3 code. file: [2aa2c3f7] check-in: [2fe579e7] user: dan branch: fts3-refactor, size: 71657
2009-11-17
12:52
Improvements to the way fts3 reads the full-text index. file: [a72c19fa] check-in: [45c051e7] user: dan branch: fts3-refactor, size: 72418
2009-11-16
16:36
Add a few extra coverage test cases for fts3. file: [4d701e31] check-in: [f29c8fca] user: dan branch: fts3-refactor, size: 72493
2009-11-14
18:41
Further OOM testing for fts3 code. Add Tcl code implementing an integrity-check for fts3. file: [1de6c9d4] check-in: [c27d46b3] user: dan branch: fts3-refactor, size: 72438
2009-11-13
17:36
Start reworking fts3 code to match the rest of SQLite (code conventions, malloc-failure handling etc.). file: [835061e6] check-in: [30a92f11] user: dan branch: fts3-refactor, size: 72213
2009-03-05
04:20
Corrected typos and misspellings. Ticket #3702. (CVS 6336) file: [35bfa67d] check-in: [6404afa0] user: shane branch: trunk, size: 226039
2009-01-14
18:59
Fix typos in comments in FTS3 implementation. (CVS 6178) file: [da5420ca] check-in: [b0f06663] user: drh branch: trunk, size: 226037
2009-01-02
01:10
Do not display matches against the right-hand side of a NOT operator in the output of the FTS snippet() or offsets() functions. (CVS 6097) file: [4e0d3b1b] check-in: [d44c84c0] user: drh branch: trunk, size: 226036
2008-12-17
15:18
Modify fts3 to support a more complex expression syntax that allows parenthesis. The new syntax is not entirely backwards compatible, so is disabled by default. Use -DSQLITE_ENABLE_FTS3_PARENTHESIS to enable it. (CVS 6034) file: [3aa6aef1] check-in: [7389b9ec] user: danielk1977 branch: trunk, size: 224631
2008-09-12
18:25
Fix the NEAR connector in FTS3 so that it can take ranges in excess of 9. The maximum range is now 32767. (CVS 5695) file: [e67453b6] check-in: [8e9b9553] user: drh branch: trunk, size: 232065
2008-07-29
01:13
Re-used prepared statement from fts3 cursor. Previously, each call to fulltextFilter() finalized any existing prepared statement and prepared a new one. In the case where idxNum has not changed, simply reseting the statement suffices. This provides an order of magnitude speedup in incoming joins against docid. (CVS 5489) file: [fa37048c] check-in: [a08a5f2b] user: shess branch: trunk, size: 232214
2008-07-15
21:32
Implement optimize() function. This merges all segments in the fts index into a single segment, including dropping delete cookies. (CVS 5417) file: [14ac2e37] check-in: [b22e187b] user: shess branch: trunk, size: 231398
2008-07-14
20:43
Delete all fts3 index data the table becomes empty. Previously, deleting all rows from an fts3 table would leave a bunch of index data describing the terms of the original data, plus deletions of those terms, perhaps with some amount of it merged together so the deletions knocked out the originals. Even when all rows were deleted that original data would hang out, though eventually it would mostly be overwritten if new data contained the same set of terms. (CVS 5413) file: [c4037314] check-in: [8b872e42] user: shess branch: trunk, size: 220403
2008-07-03
19:53
fts3 functions for testing scripts. These are a first step towards being able to write test script which verify that fts3 is internally building indices in the expected way. Both new functions are only defined if fts3.c is compiled with SQLITE_TEST defined, as when building testfixture. These functions are not intended to be part of the exposed fts3 API.    dump_terms() generates a TEXT result of all the terms in the index (or a specified segment), sorted and joined with spaces.    dump_doclist() generates a TEXT representation of the doclist associated with a given term in the index (or a specified segment). (CVS 5340) file: [192a65d3] check-in: [a48e3d95] user: shess branch: trunk, size: 218633
2008-04-12
13:06
Remove all instances of sprintf() from the FTS modules. Ticket #3049. (CVS 4996) file: [9f52ff22] check-in: [062bf5d4] user: drh branch: trunk, size: 204423
2008-02-01
15:34
Minor fixes to FTS3 so that it works better when appended to the end of the amalgamation. (CVS 4769) file: [34da63e7] check-in: [62ede669] user: drh branch: trunk, size: 204398
2007-12-13
21:54
Change all instances of "it's" in comments to either "its" or "it is", as appropriate, in case the comments are ever again read by a pedantic grammarian. Ticket #2840. (CVS 4629) file: [7d0bdfa4] check-in: [4e91a267] user: drh branch: trunk, size: 204399
2007-12-07
23:47
Change prefix search from O(N*M) to O(NlogM). The previous code linearly merged the doclists, so as the accumulated list got large, things got slow (the M term, a fucntion of the number of documents in the index). This change does pairwise merges until a single doclist remains. A test search of 't*' against a database of RFC text improves from 1m16s to 4.75s. (CVS 4599) file: [0992fca5] check-in: [feef1b15] user: shess branch: trunk, size: 204394
2007-11-24
00:41
The FTS3 amalgamation can now be appended to the SQLite amalgamation to generate a single source file that contains both components. (CVS 4558) file: [b95b4b62] check-in: [0fc61f99] user: drh branch: trunk, size: 200877
2007-11-23
17:31
Changes fts3 to use only sqlite3_malloc() and not system malloc. Ticket #2762. (CVS 4554) file: [2ea2aead] check-in: [460af6bb] user: drh branch: trunk, size: 200527
2007-11-16
00:23
Don't do anything when input doclists are both empty. Ticket #2774 (CVS 4546) file: [2e77742d] check-in: [75cb46f8] user: shess branch: trunk, size: 200030
2007-10-24
23:24
Be a bit more susicious of invalid results from the tokenizer. (CVS 4514) file: [80b5eb90] check-in: [deb8f56d] user: shess branch: trunk, size: 199999
2007-10-22
18:02
Add the NEAR operator to fts3. (CVS 4502) file: [577d6714] check-in: [aef7720e] user: danielk1977 branch: trunk, size: 199895
2007-09-13
18:16
Drop the forced error from fts3.c and add forced errors to fts2.c and fts1.c. (CVS 4427) file: [6b390b50] check-in: [fec6567a] user: shess branch: trunk, size: 189346
18:14
Add an implicit (HIDDEN) docid column. This works as an alias to rowid, similar to how things work in SQLite tables with INTEGER PRIMARY KEY. Add tests to verify operation. (CVS 4426) file: [6d266d72] check-in: [c8d23452] user: shess branch: trunk, size: 189520
18:12
Mark the table-named column HIDDEN. Add tests to make sure it's working as expected. (CVS 4425) file: [fa62f398] check-in: [ca669eaf] user: shess branch: trunk, size: 188104
2007-08-28
20:36
Fix memory leak of InteriorReader.term. Comes up when doing queries against large segments. (CVS 4315) file: [f85d5d8f] check-in: [6c617bd8] user: shess branch: trunk, size: 188097
2007-08-23
20:28
Make comments and variable naming more consistent WRT rowid versus docid/blockid. This should have no code impact. (CVS 4281) file: [c77071d5] check-in: [76f1e18e] user: shess branch: trunk, size: 188060
20:23
Fix fts3 to not have the VACUUM bug from fts2. %_content.docid is an alias to fix the rowid for documents, %_segments.blockid is an alias to fix the rowid for segment blocks. Unit test for the problem. (CVS 4280) file: [2098c9b0] check-in: [6eb2d74a] user: shess branch: trunk, size: 187769
2007-08-20
17:37
Copy fts2 to fts3, renaming, and replacing references to fts2 with fts3, including capitalization variants. (CVS 4249) file: [64ebaf64] check-in: [216c91d2] user: shess branch: trunk, size: 187003 Added