/ Artifact [1d3dee7a]

Artifact 1d3dee7a9f5a1b360a49c2310cd9ccf3c645be08bbe41ff5c3382455cf2ff981:

Shared-Schema Mode Notes

This branch contains a patch to allow SQLite connections to share schemas between database connections within the same process in order to save memory. Schemas may be shared between multiple databases attached to the same or distinct connection handles.

To activate shared-schemas, a database connection must be opened using the sqlite3_open_v2() API with the SQLITE_OPEN_SHARED_SCHEMA flag specified. The main database and any attached databases will then share an in-memory Schema object with any other database opened within the process for which:

Temp databases (those populated with "CREATE TEMP TABLE" and similar statements) never share schemas.

Connections opened with the SQLITE_OPEN_SHARED_SCHEMA flag specified may not modify any database schema except that belonging to the temp database in anyway. This includes creating or dropping database objects, vacuuming the database, or running ANALYZE when the sqlite_stat[14] tables do not exist.

For SQLITE_OPEN_SHARED_SCHEMA connections, the SQLITE_DBSTATUS_SCHEMA_USED sqlite3_db_status() verb distributes the memory used for a shared schema object evenly between all database connections that share it.

The ".shared-schema" Command

The shell tool on this branch contains a special dot-command to help with managing databases. The ".shared-schema" dot-command can be used to test whether or not two databases are similar enough to share in-memory schemas, and to fix minor problems that prevent them from doing so. To test if two or more database are compatible, one database is opened directly using the shell tool and the following command issued:

    .shared-schema check <database-1> [<database-2>]...

where <database-1;> etc. are replaced with the names of database files on disk. For each database specified on the command line, a single line of output is produced. If the database can share an in-memory schema with the main database opened by the shell tool, the output is of the form:

    <database> is compatible

Otherwise, if the database cannot share a schema with the main db, the output is of the form:

    <database> is NOT compatible (<reason>)

where <reason> indicates the cause of the incompatibility. <reason> is always one of the following.

The final three problems in the list above can be fixed using the .shared-schema command. To modify such a database so that it can share a schema with the main database, the following shell command is used:

    .shared-schema fix <database-1> [<database-2>]...

If a database can be modified so that it may share a schema with the main database opened by the shell tool, output is as follows:

    Fixing <database>... <database> is compatible

If a database does not require modification, or cannot be modified such that it can share a schema with the main database, the output of "fix" is identical to that of the "check" command.

Implementation Notes

A single Schema object is never used by more than one database simultaneously, regardless of whether or not those databases are attached to the same or different database handles. Instead, a pool of schema objects is maintained for each unique sqlite_master-contents/schema-cookie combination opened within the process. Each time database schemas are required by a connection, for example as part of an sqlite3_prepare*(), sqlite3_blob_open() or sqlite3_blob_open() call, it obtains the minimum number of schemas required from the various schema-pools, returning them at the end of the call. This means that a single schema-pool only ever contains more than one copy of the schema if:

The size of a schema-pool never shrinks. Each schema pool always maintains a number of schema objects equal to the highwater mark of schema objects simultaneously required by clients.

This approach is preferred to allowing multiple databases to use the same Schema object simultaneously for three reasons:

SQLITE_OPEN_SHARED_SCHEMA connections do not store their virtual-table handles in the Table.pVTable list of each table. This would not work, as (a) there is no guarantee that a connection will be assigned the same Schema object each time it requests one from a schema-pool and (b) a single Schema (and therefore Table) object may correspond to tables in two or more databases attached to a single connection. Instead, all virtual-table handles associated with a single database are stored in a linked-list headed at Db.pVTable.