SQL Lite DB drawbacks then going for full text index library

(1) By anonymous on 2021-04-22 16:38:03 [source]

Hi,

I have 5,000 txt files. I want to create a mobile app to do full text search

When i looked for it few suggested the following

a) use sqlite fts5 (or) b) search-index library

I'm now sure what the drawbacks and how much slower sqlite would be when to do a full text search for 5,000 files which can be converted to 5,00,000 records as each file would have 100 paragraphs and each paragraph can be saved as a row

Is it worth to put 5 lakhs records in SQLite and do a full text search. Will it be faster?

Please let me know in detail

Thanks

(2) By warmwaffles on 2021-04-22 16:49:36 in reply to 1 [link] [source]

It depends on your access pattern or what you are trying to accomplish. Yes you can store each paragraph as a separate row and have a document identifier for each row to keep them grouped together.

Personally, I'd keep each document as one record. 5,000 rows with a couple hundred paragraphs each is not that bad.

Do you have any initial benchmarks that you can share?