SQLite Forum

Truncated Words bug with fts5vocab
Login
Hi there; new to the forum.

Can anyone recreate this bug I have found with FTS5?

Steps:

1. Load FTS5 Extension, if not included in the build.

2. Create a normal table with at least two columns called 'name' and 'keywords':

CREATE TABLE content (name TEXT, keywords TEXT);

3. Create a fts table based on that table:

CREATE VIRTUAL TABLE content_fts
USING fts5(name, keywords, content='content', tokenize='porter ascii');

4. Create a fts5vocab table based on that in turn:

CREATE VIRTUAL TABLE content_vocab
USING fts5vocab('content_fts','row');

5. Add a trigger to insert data into the fts table automatically:

CREATE TRIGGER content_ai AFTER INSERT ON content
BEGIN
	INSERT INTO content_fts (name, keywords) 
	VALUES (new.name, new.keywords); 
END;

6. Insert data containing the words 'Sample' and 'Wallpaper' into the table:

INSERT INTO content VALUES ('Sample Gif', 'content,sample,gif');
INSERT INTO content VALUES ('Sample Wallpaper', 'content,sample,wallpaper');

7. Query the content_vocab table,to see 'sample' and 'wallpaper' are truncated:

SELECT * FROM content_vocab

content	2	2
gif	1	2
sampl	2	4
wallpap	1	2