SQLite Forum

Inserting One Billion Rows in SQLite Under A Minute
Login
> My experience with making the page cache very large is it adds a big delay when you close the DB (presumably due to the I/O flushing the cache).

It takes no longer to free(x) if x is 1 byte or if x is 1 TB.  

What you are talking about is "flushing dirty pages" (a commit operation).  It is related to the number of dirty pages in the cache.  Writing 1000 pages takes pretty much the same total time actually writing whether you write them one by each spread over an hour or all at once.

 > If the schema has several indices, is it better to defer adding them until after the tables are loaded?

Maybe.  If you can do that then yes, the number of pages to be updated decreases as the size of the batch increases.

 > I don't know if an index build/rebuild can optimize page cache management while walking a large table for an index build/rebuild.

Index Pages are also stored in the page cache.  If the page cache is too small then "partial changes" must be spilled onto physical media more frequently and these I/O operations take time.  The fewer I/O operations you can perform on the same page (by keeping it in the closest cache -- the application cache) the better.