Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
History for ext/fts3/unicode/mkunicode.tcl
|
2025-02-27
| ||
| 21:17 | [cbf5f7b5c8] part of check-in [f50c21484d] Approximately 100 typo corrections spanning the whole tree, submitted via forum post 0db9827f0464bc33 and individually audited and verified. Affects only code comments, innocuous test strings, error message text in tool (not library) code, and configure-level help text. (check-in: [f50c21484d] user: stephan branch: trunk, size: 25558) | |
|
2024-08-17
| ||
| 17:22 | [63db9624cc] part of check-in [8f9257361b] Add tests to restore coverage of fts5_tokenizer.c. (check-in: [8f9257361b] user: dan branch: fts5-locale, size: 25557) | |
|
2020-11-26
| ||
| 20:13 | [d5aebf022f] part of check-in [326d579d77] Update mkunicode.tcl to match the change erroneously made to machine generated file fts5_unicode2.c in [b7b7bde9]. (check-in: [326d579d77] user: dan branch: trunk, size: 25519) | |
|
2020-07-29
| ||
| 16:18 | [956447806e] part of check-in [a80ae2c98b] Dozens and dozens of typo fixes in comments. This change adds no value to the end product and is disruptive, so it is questionable whether or not it will ever land on trunk. (check-in: [a80ae2c98b] user: drh branch: typos, size: 25445) | |
|
2019-03-20
| ||
| 05:45 | [bf7fcaa6d6] part of check-in [1c0fe5b576] Fix various harmless compiler warnings seen with MSVC. (check-in: [1c0fe5b576] user: mistachkin branch: noWarnings, size: 25443) | |
|
2019-01-02
| ||
| 23:49 | [49499f7964] part of check-in [703029ac6d] Fix harmless compiler warnings in the unicode2 logic of FTS3 and FTS5. (check-in: [703029ac6d] user: drh branch: trunk, size: 25439) | |
|
2018-12-28
| ||
| 07:37 | [2315b3f8c8] part of check-in [c564bf8701] Fix problems in fts5 found by ASAN. (check-in: [c564bf8701] user: dan branch: trunk, size: 25422) | |
|
2018-12-03
| ||
| 17:40 | [2ea30d8122] part of check-in [7149dacf1d] Remove the unused sqlite3Fts5UnicodeNCat() function. (check-in: [7149dacf1d] user: drh branch: trunk, size: 25353) | |
| 16:14 | [106bb4ff63] part of check-in [06177f3f11] Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5 and FTS3/4. (check-in: [06177f3f11] user: dan branch: trunk, size: 25420) | |
|
2018-07-13
| ||
| 19:52 | [0069320b64] part of check-in [80d2b9e635] Add the "categories" option to the unicode61 tokenizer in fts5. (check-in: [80d2b9e635] user: dan branch: trunk, size: 25078) | |
|
2017-03-20
| ||
| 18:53 | [ab0543a3b2] part of check-in [16a8e84fa7] Fix some problems in fts3 found by address-sanitizer. (check-in: [16a8e84fa7] user: dan branch: trunk, size: 18339) | |
|
2016-02-12
| ||
| 18:48 | [2debed3f58] part of check-in [a049fbbde5] Fix a potential buffer overread provoked by invalid utf-8 in fts5. (check-in: [a049fbbde5] user: dan branch: trunk, size: 18325) | |
|
2015-07-02
| ||
| 15:52 | [95cf7ec186] part of check-in [7819002ed8] Remove "#ifdef SQLITE_ENABLE_FTS5" from individual fts5 source files. Add a single "#if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5)" to fts5.c. (check-in: [7819002ed8] user: dan branch: trunk, size: 18297) | |
|
2015-05-23
| ||
| 15:43 | [ed0534dd51] part of check-in [5165de548b] Avoid making redundant copies of position-lists within the fts5 code. (check-in: [5165de548b] user: dan branch: fts5, size: 18368) | |
|
2015-05-22
| ||
| 06:08 | [b321eea0c1] part of check-in [fea8a4db9d] Improve test coverage of fts5_unicode2.c. (check-in: [fea8a4db9d] user: dan branch: fts5, size: 18386) | |
|
2015-02-02
| ||
| 11:32 | [159c1194da] part of check-in [fb10bbb9f9] Fix some problems with building fts5 and fts3 together using the amalgamation. (check-in: [fb10bbb9f9] user: dan branch: fts5, size: 22145) | |
|
2015-01-01
| ||
| 18:03 | [4199cb8870] part of check-in [4b3651677e] Merge latest trunk changes with this branch. (check-in: [4b3651677e] user: dan branch: fts5, size: 22028) | |
| 16:46 | [2fa92b916b] part of check-in [d09f7800cf] Add a version of the unicode61 tokenizer to fts5. (check-in: [d09f7800cf] user: dan branch: fts5, size: 22032) | |
|
2014-08-06
| ||
| 18:50 | [a2567f9d6a] part of check-in [bcf6d775f9] A couple more harmless compiler warnings eliminated. (check-in: [bcf6d775f9] user: drh branch: trunk, size: 21588) | |
| 17:49 | [ddeb6629fb] part of check-in [a2a60307ea] Fix two more harmless compiler warnings. Make sure the fts3_unicode2.c file is in sync with mkunicode.tcl. (check-in: [a2a60307ea] user: drh branch: trunk, size: 21585) | |
|
2013-06-05
| ||
| 16:17 | [dc6f268eb5] part of check-in [6cfd9af525] Up until now the fts4 "unicode61" tokenizer has treated all private use codepoints except the first and last of each of the three ranges as alphanumeric (eligible to be part of tokens). This commit fixes this so that all private use codepoints are considered alphanumeric. In other words, it fixes the handling of codepoints 0xE000, 0xF8FF, 0xF0000, 0xFFFFD, 0x100000 and 0x10FFFD. (check-in: [6cfd9af525] user: dan branch: trunk, size: 21592) | |
|
2012-06-06
| ||
| 19:30 | [7a9bc018e2] part of check-in [790f76a589] Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0". (check-in: [790f76a589] user: dan branch: trunk, size: 21525) | |
|
2012-05-28
| ||
| 12:22 | [2029991cc2] part of check-in [c00bb5d460] Omit the fts3 unicode character class routines from the build if fts3/4 is disabled. (check-in: [c00bb5d460] user: drh branch: trunk, size: 15497) | |
|
2012-05-26
| ||
| 18:28 | [de64862abd] part of check-in [e71495a817] If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer. (check-in: [e71495a817] user: dan branch: fts4-unicode, size: 15339) | |
| 17:57 | [2775280062] part of check-in [b89d3834f6] Change the format of the tables used by sqlite3FtsUnicodeTolower() to make them a little smaller. (check-in: [b89d3834f6] user: dan branch: fts4-unicode, size: 15228) | |
|
2012-05-25
| ||
| 19:50 | [a7214d1705] part of check-in [cf7b25d476] Add special fast paths to sqlite3FtsUnicodeTolower() and Isalnum() for codepoints in the ASCII range. (check-in: [cf7b25d476] user: dan branch: fts4-unicode, size: 14223) | |
| 18:48 | [3ff244e412] part of check-in [3dc567ef47] Fix comments in generated file fts3_unicode2.c. (check-in: [3dc567ef47] user: dan branch: fts4-unicode, size: 13497) | |
| 17:50 | Added: [1f50ed0021] part of check-in [0c13570ec7] Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators. (check-in: [0c13570ec7] user: dan branch: fts4-unicode, size: 11733) | |