I had a program some years ago that did a very similar pattern, and the problem is that that version of FTS needs to create some really large blobs when merging FTS nodes. I don't remember all the details, but my solution at the time was to hand shard the data to keep the size of FTS tables smaller, then run multiple queries to gather the same data. [](http://sqlite.1065341.n5.nabble.com/Segfault-during-FTS-index-creation-from-huge-data-td81465.html) seems to be a link to a six year old thread on the same or very similar / adjacent topic.