SQLite Forum

sqlite write performance is slower than the filesystem
Login
If I'm reading [your Python][prog] correctly, you seem to be trying to do
all of the following on every write:

  1.  Open an SQLite database connection.
  2.  Configure that connection.
  3.  Do the write.
  4.  Vacuum.
  5.  Close the connection.

I suspect you are spending most of the CPU time in steps 1, 2, 4, and 5.
I suggest you revise your program so that it opens a single database connection,
just once, configures it just once, then do lots of writes.  Then at the very
end, vacuum if you think you should (you probably don't need to) and then
close the connection.  I think that will go *a lot* faster.

[prog]: https://gist.github.com/bioinfornatics/2870d9e7dba192114f32ce8299796269