SQLite Forum

Question about memory management
Login
Firstly, hello. I'm brand new to SQLite but 3+ decades of coding in Clarion from Soft Velocity.

I implemented SQLite as I was told it is fast, and that it is. It would be perfect for my application save for one problem. As I import csv files into databases - one for each subject I am importing - my computer memory fills right up. All 64 GBs get used and the system crashes. I'm running 30 programs at once and each is doing an import and there are millions of records per import. I can manage 5 imports at once to be safe. I have tried closing the database every 20,000 records to free up buffers, but that has not helped. Meanwhile I am back to Btrieve which is working but slower. 

Lastly, one other perplexing issue is that the import process self terminates between 40% and 65% of the way through the imports with SQLite, but with Pervasive, the imports complete. Any comments?

SQLite has so many advantages that I would really like to resolve these problems Also, I love that I can distribute my app without buying another Btrieve license.

Can anyone enlighten me as to what is going on with memory. My first thought is that SQLITE_DEFAULT_MEMSTATUS being set to zero is the culprit, but, like I say, I'm brand new.