Offsets function for FTS5, any ready-to-use or sample implementation?

(1) By Edwin Yip (eysqlite) on 2021-02-01 06:39:04 [link]

Hello,

I'm currently using FTS4 and would like to use FTS5 to take advantage of its partial matching ability.

With FTS4 I use the `offsets` function to locate the exact position in the source documents. But since FTS5 has no such function, I'd like to ask:

- I read from the SQLite document that the `offsets` function might be added in the future, so what's the estimated time frame, if any?

- Anyone can point me to any custom `offsets` implementations for FTS5 so that i can port it to my language? 

Take care and Thanks!

(2) By Dan Kennedy (dan) on 2021-02-01 16:49:14 in reply to 1

> I read from the SQLite document that the offsets function might be added in the future, so what's the estimated time frame, if any?

Huh. Where does it say that?

> Anyone can point me to any custom offsets implementations for FTS5 so that i can port it to my language?

I don't know of one. You would need to create a function like this one:

[](https://sqlite.org/fts5.html#custom_auxiliary_functions)

That uses the xInstCount(), xInst(), xColumnText() and xTokenize() methods to figure out the byte offsets of each matching phrase:

[](https://sqlite.org/fts5.html#xTokenize)

I think the implementation would have a lot in common with the implementation of the built-in highlight() function:

[](https://sqlite.org/src/info/f558e1fb9f0?ln=108-247)

(3) By Edwin Yip (eysqlite) on 2021-02-02 04:57:18 in reply to 2 [link]

Hello Dan,

Thanks for the helpful pointers!

> > I read from the SQLite document that the offsets function might be added in the future, so what's the estimated time frame, if any?

> Huh. Where does it say that?

I found this: "The set of built-in auxiliary functions provided by FTS5 may be improved upon in the future." from https://www.sqlite.org/fts5.html#_auxiliary_function_changes_