SQLite Forum

memory vs mmap
Hi Keith. Thanks for your thoughts.

The use scenario is a machine with sufficient memory to hold the entire database in memory, but lots of processes accessing it, and insufficient memory if all of them expect a private copy of the read-only database.

I expected mmap() to load the entire database into virtual memory, and I didn't think it was important whether that happened immediately at startup or over time. If that is an issue, maybe what I need is an Sqlite modification that after the mmap it accesses the entire file to ensure it has all been loaded into memory upfront.

Note that the load time of copying the database from disk to memory, even if it needed to be done by every process, is not a problem. Fast startup is not required.

Are you suggesting a ramdisk would be faster, even if it goes through the same mmap() process? (ignoring the small once-off cost of loading from disk versus loading from RAM).

I think I might have stuffed up in my original message - the performance difference is running the actual long-running application, not reading the 1.6 GB database into memory.

I'm not familiar with Sqlite internals, so I assume the performance difference is being caused by the internal XFetch() or whatever calls being slower than whatever ":memory:" is using. I actually assumed that both mmap() and memory would make the same XFetch() calls so there would be no difference in speed, but that is not the case for some reason. That's what I would like to understand.

I'd also like to understand the technical restriction preventing :memory: databases from being shared across processes, at least for read-only. It's all the exact same data.