SQLite Forum

persistent in-memory database
Login

persistent in-memory database

(1) By Naveed Ul Mustafa (naveedum) on 2021-12-15 15:35:26 [link] [source]

I am reading the book "SQLite Database system design and implementation" by Sibsankar Haldar.

It states (in summary of Chapter 3) "SQLite also supports in-memory databases. These databases are created and initialized when an application opens them, and they are deleted when application closes them".

I want to create a persistent database. One way I can think of is forcing the placement of database file in the persistent memory (for example, intel Optane DIMM). However, if the application closes the database (file placed in persistent memory) what will happen to the database? Will SQLite delete/Zero-out the file in persistent memory or not?

Thank you for the guidance.

(2) By ddevienne on 2021-12-15 16:24:31 in reply to 1 [link] [source]

There's no such thing as persistent memory in standard C/C++.

So either that memory is exposed as pseudo-files, and thus acts as very fast files.
Or it requires use of non-standard APIs, and you must use a custom VFS to use it.

SQLite memory databases are dynamically allocated, in transient/volatile memory.

(3) By Warren Young (wyoung) on 2021-12-15 18:47:56 in reply to 1 [source]

There are two kinds of Optane storage.

One kind is "Optane SSD", and as far as I know, it just looks like a regular block device: put a filesystem on it, mount it, and use SQLite as normal.

The other kind is a form of NVDIMM, which the OS kernel handles differently. See for instance the RHEL docs on this, which should apply to other Linuxes of the same generation or later.

I don't know if it's legal to put a filesystem onto an NVDIMM device. If so, then you can treat it like a special type of SSD. If not, then we have to ask if SQLite supports raw device I/O.

You certainly cannot use SQLite with raw devices in WAL mode, since that wouldn't permit SQLite to store the separate WAL and SHM files.

It might be possible to hack the traditional I/O mode of SQLite to use a raw device, though.