Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
History for ext/fts5/fts5_tokenize.c
|
2019-04-13
| ||
| 04:38 | [2e508c6a] part of check-in [07ee06fd] Use the 64-bit memory allocator interfaces in extensions, whenever possible. (check-in: [07ee06fd] user: drh branch: trunk, size: 34630) | |
|
2019-01-08
| ||
| 20:02 | [4d904c23] part of check-in [ca67f2ec] Use 64-bit math to compute the sizes of memory allocations in extensions. (check-in: [ca67f2ec] user: drh branch: trunk, size: 34628) | |
|
2018-12-31
| ||
| 21:43 | [8b7ef00c] part of check-in [b57c545a] Fix harmless compiler warnings. (check-in: [b57c545a] user: drh branch: trunk, size: 34557) | |
|
2018-12-28
| ||
| 14:33 | [d49f479c] part of check-in [c3a3a111] Avoid an undefined left-shift operation in fts5 caused by malformed utf-8 text. (check-in: [c3a3a111] user: dan branch: trunk, size: 34554) | |
| 07:37 | [240f849c] part of check-in [c564bf87] Fix problems in fts5 found by ASAN. (check-in: [c564bf87] user: dan branch: trunk, size: 34559) | |
|
2018-12-03
| ||
| 16:14 | [ca2b6a03] part of check-in [06177f3f] Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5 and FTS3/4. (check-in: [06177f3f] user: dan branch: trunk, size: 34549) | |
|
2018-07-13
| ||
| 19:52 | [ebd13d03] part of check-in [80d2b9e6] Add the "categories" option to the unicode61 tokenizer in fts5. (check-in: [80d2b9e6] user: dan branch: trunk, size: 34058) | |
|
2016-02-11
| ||
| 17:01 | [2ce7b441] part of check-in [bc3f7900] Handle parser stack overflow when parsing fts5 query expressions. Fix some compiler warnings in fts5 code. (check-in: [bc3f7900] user: dan branch: trunk, size: 33283) | |
|
2016-01-23
| ||
| 18:51 | [4d5c4f18] part of check-in [72d53699] Fix an fts5 problem with using both xPhraseFirst() and xPhraseFirstColumn() within a single statement in detail=col mode. (check-in: [72d53699] user: dan branch: fts5-perf, size: 33170) | |
|
2015-12-23
| ||
| 16:42 | [504984ac] part of check-in [5d44d4a6] Fix some harmless gcc compiler warnings. Mostly in fts5, but also two in the core code. (check-in: [5d44d4a6] user: dan branch: trunk, size: 33175) | |
|
2015-12-16
| ||
| 23:30 | [618efe03] part of check-in [1d0e6aa1] Fix even more harmless compiler warnings. (check-in: [1d0e6aa1] user: mistachkin branch: msvcWarn, size: 33187) | |
|
2015-10-14
| ||
| 20:34 | [12c5d925] part of check-in [1c46c194] Fix harmless compiler warnings. (check-in: [1c46c194] user: mistachkin branch: trunk, size: 33172) | |
|
2015-09-02
| ||
| 19:48 | [f380f46f] part of check-in [bdedd838] Further tests to raise coverage of fts5 synonym code to 100%. Fix a dropped error code in the same. (check-in: [bdedd838] user: dan branch: fts5-incompatible, size: 33167) | |
|
2015-08-29
| ||
| 15:44 | [71054151] part of check-in [fc718684] Another change to the fts5 tokenizer API. (check-in: [fc718684] user: dan branch: fts5-incompatible, size: 33174) | |
|
2015-08-28
| ||
| 19:56 | [07a89441] part of check-in [90b85b42] Change the fts5 tokenizer API to allow more than one token to occupy a single position within a document. (check-in: [90b85b42] user: dan branch: fts5-incompatible, size: 33224) | |
|
2015-07-31
| ||
| 14:43 | [2836f672] part of check-in [c3c672af] Fix a bug in the fts5 porter tokenizer preventing it from passing xCreate() arguments through to its parent tokenizer. (check-in: [c3c672af] user: dan branch: trunk, size: 33071) | |
|
2015-07-02
| ||
| 15:52 | [30f97a8c] part of check-in [7819002e] Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c. (check-in: [7819002e] user: dan branch: trunk, size: 32972) | |
|
2015-05-30
| ||
| 11:49 | [97251d68] part of check-in [e008c3c8] Remove the "#include sqlite3Int.h" from fts5Int.h. (check-in: [e008c3c8] user: dan branch: fts5, size: 33045) | |
|
2015-05-22
| ||
| 06:08 | [24649425] part of check-in [fea8a4db] Improve test coverage of fts5_unicode2.c. (check-in: [fea8a4db] user: dan branch: fts5, size: 33082) | |
|
2015-05-20
| ||
| 09:27 | [6f4d2cbe] part of check-in [0e91a6a5] Improve test coverage of fts5_tokenize.c. (check-in: [0e91a6a5] user: dan branch: fts5, size: 33260) | |
|
2015-05-19
| ||
| 11:32 | [4d9d5047] part of check-in [de9f8ef6] Fix a memory leak that could follow an OOM condition in fts5. (check-in: [de9f8ef6] user: dan branch: fts5, size: 33245) | |
|
2015-04-29
| ||
| 20:54 | [830eae0d] part of check-in [c1f07a3a] Improve fts5 tests. (check-in: [c1f07a3a] user: dan branch: fts5, size: 33150) | |
|
2015-03-11
| ||
| 14:51 | [c07f2c2f] part of check-in [f5db4892] Add an optimization to the fts5 unicode tokenizer code. (check-in: [f5db4892] user: dan branch: fts5, size: 33078) | |
|
2015-03-04
| ||
| 08:29 | [c3fe3091] part of check-in [a5d5468c] Fix a couple of build problems. (check-in: [a5d5468c] user: dan branch: fts5, size: 33019) | |
|
2015-02-02
| ||
| 11:32 | [0d108148] part of check-in [fb10bbb9] Fix some problems with building fts5 and fts3 together using the amalgamation. (check-in: [fb10bbb9] user: dan branch: fts5, size: 32945) | |
|
2015-01-17
| ||
| 17:48 | [7c61d5c3] part of check-in [96ea6004] Improve the performance of the fts5 porter tokenizer implementation. (check-in: [96ea6004] user: dan branch: fts5, size: 32932) | |
|
2015-01-12
| ||
| 17:58 | [bdb6a1f5] part of check-in [f22dbcca] Optimize the unicode61 tokenizer so that it handles ascii text faster. Make it the default tokenizer. Change the name of the simple tokenizer to "ascii". (check-in: [f22dbcca] user: dan branch: fts5, size: 24110) | |
|
2015-01-06
| ||
| 19:08 | [4c30cf32] part of check-in [65f0262f] Remove the iPos parameter from the tokenizer callback. Fix the "tokenchars" and "separators" options on the simple tokenizer. (check-in: [65f0262f] user: dan branch: fts5, size: 22293) | |
|
2015-01-01
| ||
| 16:46 | [5a0ad464] part of check-in [d09f7800] Add a version of the unicode61 tokenizer to fts5. (check-in: [d09f7800] user: dan branch: fts5, size: 21276) | |
|
2014-12-29
| ||
| 11:24 | [5d6e7853] part of check-in [b33fe0dd] Fixes to built-in tokenizers. (check-in: [b33fe0dd] user: dan branch: fts5, size: 12914) | |
|
2014-11-15
| ||
| 20:07 | [8360c0d1] part of check-in [fba0b5fc] Fix the customization interfaces so that they match the documentation. (check-in: [fba0b5fc] user: dan branch: fts5, size: 3561) Added | |