|
|
Requirements for the SQLite Database File Format
This document contains detailed requirements for the database file format and the file I/O characteristics of SQLite.
H30010: The system shall ensure that at the successful conclusion of a database transaction the contents of the database file constitute a well-formed SQLite database file.
H30020: The system shall ensure that at the successful conclusion of a database transaction the contents of the database file are a valid serialization of the contents of the logical SQL database produced by the transaction.
H30030: The first 16 bytes of a well-formed database file shall contain the UTF-8 encoding of the string "SQLite format 3" followed by a single nul-terminator byte.
H30040: The 6 bytes beginning at byte offset 18 of a well-formed database image shall contain the values 0x01, 0x01, 0x00, 0x40, 0x20 and 0x20, respectively.
H30120: The 4-byte big-endian signed integer field at byte offset 44 of a well-formed database image, the schema layer file format field, shall be set to an integer value between 1 and 4, inclusive.
H30130: The 4-byte big-endian unsigned integer field at byte offset 48 of a well-formed database image shall be set to the value of the database default page-cache size.
H30140: If the database is not an auto-vacuum capable database, then the 4 byte big-endian unsigned integer field at byte offset 52 of a well-formed database image shall contain the value 0.
H30141: If the database is an auto-vacuum capable database, then the 4 byte big-endian unsigned integer field at byte offset 52 of a well-formed database image shall contain the numerically largest root-page number of any table or index B-Tree within the database image.
H30150: The 4 byte big-endian unsigned integer field at byte offset 56 of a well-formed database image shall be set to 1 if the database text-encoding is UTF-8, 2 if the database text-encoding is little-endian UTF-16, and 3 if the database text-encoding is big-endian UTF-16.
H30160: The 4-byte big-endian unsigned integer field at byte offset 60 of a well-formed database image shall be set to the value of the database user-cookie.
H30190: The 2-byte big-endian unsigned integer field at byte offset 16 of a well-formed database image shall be set to the value of the database page-size.
H30191: The page-size of an SQLite database in bytes shall be an integer power of 2 between 512 and 32768, inclusive.
H30170: The 4-byte big-endian unsigned integer field at byte offset 64 of a well-formed database image shall be set to the value of the database incremental-vacuum flag.
H30171: The value of the incremental-vacuum flag of an SQLite database shall be either 0 or 1.
H30180: In a well-formed non-autovacuum database (one with a zero stored in the 4-byte big-endian integer value beginning at byte offset 52 of the database file header, the incremental vacuum flag is set to 0.
H30200: The size of a well formed database image shall be an integer multiple of the database page size.
H30210: Each page of a well formed database image shall be exactly one of a B-Tree page, an overflow page, a free page, a pointer-map page or the locking page.
H30220: The database page that starts at byte offset 230, the locking page, shall never be used for any purpose.
H30230: In a well-formed database file, the portion of the first database page not consumed by the database file-header (all but the first 100 bytes) contains the root node of a table B-Tree, the schema table.
H30240: All records stored in the schema table contain exactly five fields.
H30250: For each SQL table in the database apart from itself ("sqlite_master"), the schema table of a well-formed database file contains an associated record.
H30260: The first field of each schema table record associated with an SQL table shall be the text value "table".
H30270: The second field of each schema table record associated with an SQL table shall be a text value set to the name of the SQL table.
H30280: In a well-formed database file, the third field of all schema table records associated with SQL tables shall contain the same value as the second field.
H30290: In a well-formed database file, the fourth field of all schema table records associated with SQL tables that are not virtual tables contains the page number (an integer value) of the root page of the associated table B-Tree structure within the database file.
H30300: If the associated database table is a virtual table, the fourth field of the schema table record shall contain the integer value 0 (zero).
H30310: In a well-formed database, the fifth field of all schema table records associated with SQL tables shall contain a "CREATE TABLE" or "CREATE VIRTUAL TABLE" statment (a text value). The details of the statement shall be such that executing the statement would create a table of precisely the same name and schema as the existing database table.
H30320: For each PRIMARY KEY or UNIQUE constraint present in the definition of each SQL table in the database, the schema table of a well-formed database shall contain a record with the first field set to the text value "index", and the second field set to a text value containing a string of the form "sqlite_autoindex_<name>_<idx>", where <name> is the name of the SQL table and <idx> is an integer value.
H30330: In a well-formed database, the third field of all schema table records associated with SQL PRIMARY KEY or UNIQUE constraints shall contain the name of the table to which the constraint applies (a text value).
H30340: In a well-formed database, the fourth field of all schema table records associated with SQL PRIMARY KEY or UNIQUE constraints shall contain the page number (an integer value) of the root page of the associated index B-Tree structure.
H30350: In a well-formed database, the fifth field of all schema table records associated with SQL PRIMARY KEY or UNIQUE constraints shall contain an SQL NULL value.
H30360: For each SQL index in the database, the schema table of a well-formed database shall contain a record with the first field set to the text value "index" and the second field set to a text value containing the name of the SQL index.
H30370: In a well-formed database, the third field of all schema table records associated with SQL indexes shall contain the name of the SQL table that the index applies to.
H30380: In a well-formed database, the fourth field of all schema table records associated with SQL indexes shall contain the page number (an integer value) of the root page of the associated index B-Tree structure.
H30390: In a well-formed database, the fifth field of all schema table records associated with SQL indexes shall contain an SQL "CREATE INDEX" statement (a text value). The details of the statement shall be such that executing the statement would create an index of precisely the same name and content as the existing database index.
H30400: For each SQL view in the database, the schema table of a well-formed database shall contain a record with the first field set to the text value "view" and the second field set to a text value containing the name of the SQL view.
H30410: In a well-formed database, the third field of all schema table records associated with SQL views shall contain the same value as the second field.
H30420: In a well-formed database, the third field of all schema table records associated with SQL views shall contain the integer value 0.
H30430: In a well-formed database, the fifth field of all schema table records associated with SQL indexes shall contain an SQL "CREATE VIEW" statement (a text value). The details of the statement shall be such that executing the statement would create a view of precisely the same name and definition as the existing database view.
H30440: For each SQL trigger in the database, the schema table of a well-formed database shall contain a record with the first field set to the text value "trigger" and the second field set to a text value containing the name of the SQL trigger.
H30450: In a well-formed database, the third field of all schema table records associated with SQL triggers shall contain the name of the database table or view to which the trigger applies.
H30460: In a well-formed database, the third field of all schema table records associated with SQL triggers shall contain the integer value 0.
H30470: In a well-formed database, the fifth field of all schema table records associated with SQL indexes shall contain an SQL "CREATE TRIGGER" statement (a text value). The details of the statement shall be such that executing the statement would create a trigger of precisely the same name and definition as the existing database trigger.
H30480: In an auto-vacuum database, all pages that occur before the page number stored in the auto-vacuum last root-page field of the database file header (see H30140) must be either B-Tree root pages, pointer-map pages or the locking page.
H30490: In an auto-vacuum database, no B-Tree root pages may occur on or after the page number stored in the auto-vacuum last root-page field of the database file header (see H30140) must be either B-Tree root pages, pointer-map pages or the locking page.
H30500: As well as the schema table, a well-formed database file contains N table B-Tree structures, where N is the number of non-virtual tables in the logical database, excluding the sqlite_master table but including sqlite_sequence and other system tables.
H30510: A well-formed database file contains N table B-Tree structures, where N is the number of indexes in the logical database, including indexes created by UNIQUE or PRIMARY KEY clauses in the declaration of SQL tables.
H30520: A 64-bit signed integer value stored in variable length integer format consumes from 1 to 9 bytes of space.
H30530: The most significant bit of all bytes except the last in a serialized variable length integer is always set. Unless the serialized form consumes the maximum 9 bytes available, then the most significant bit of the final byte of the representation is always cleared.
H30540: The eight least significant bytes of the 64-bit twos-compliment representation of a value stored in a 9 byte variable length integer are stored in the final byte (byte offset 8) of the serialized variable length integer. The other 56 bits are stored in the 7 least significant bits of each of the first 8 bytes of the serialized variable length integer, in order from most significant to least significant.
H30550: A variable length integer that consumes less than 9 bytes of space contains a value represented as an N-bit unsigned integer, where N is equal to the number of bytes consumed by the serial representation (between 1 and 8) multiplied by 7. The N bits are stored in the 7 least significant bits of each byte of the serial representation, from most to least significant.
H30560: A database record consists of a database record header, followed by database record data. The first part of the database record header is a variable length integer containing the total size (including itself) of the header in bytes.
H30570: Following the length field, the remainder of the database record header is populated with N variable length integer fields, where N is the number of database values stored in the record.
H30580: Following the database record header, the database record data is made up of N variable length blobs of data, where N is again the number of database values stored in the record. The n blob contains the data for the nth value in the database record. The size and format of each blob of data is encoded in the corresponding variable length integer field in the database record header.
H30590: A value of 0 stored within the database record header indicates that the corresponding database value is an SQL NULL. In this case the blob of data in the data area is 0 bytes in size.
H30600: A value of 1 stored within the database record header indicates that the corresponding database value is an SQL integer. In this case the blob of data contains the integer value, formatted as a 1-byte big-endian signed integer.
H30610: A value of 2 stored within the database record header indicates that the corresponding database value is an SQL integer. In this case the blob of data contains the integer value, formatted as a 2-byte big-endian signed integer.
H30620: A value of 3 stored within the database record header indicates that the corresponding database value is an SQL integer. In this case the blob of data contains the integer value, formatted as a 3-byte big-endian signed integer.
H30630: A value of 4 stored within the database record header indicates that the corresponding database value is an SQL integer. In this case the blob of data contains the integer value, formatted as a 4-byte big-endian signed integer.
H30640: A value of 5 stored within the database record header indicates that the corresponding database value is an SQL integer. In this case the blob of data contains the integer value, formatted as a 6-byte big-endian signed integer.
H30650: A value of 6 stored within the database record header indicates that the corresponding database value is an SQL integer. In this case the blob of data contains the integer value, formatted as a 8-byte big-endian signed integer.
H30660: A value of 7 stored within the database record header indicates that the corresponding database value is an SQL real (floating point number). In this case the blob of data contains an 8-byte IEEE floating point number, stored in big-endian byte order.
H30670: A value of 8 stored within the database record header indicates that the corresponding database value is an SQL integer, value 0. In this case the blob of data in the data area is 0 bytes in size.
H30680: A value of 9 stored within the database record header indicates that the corresponding database value is an SQL integer, value 1. In this case the blob of data in the data area is 0 bytes in size.
H30690: An even value greater than or equal to 12 stored within the database record header indicates that the corresponding database value is an SQL blob field. The blob of data contains the value data. The blob of data is exactly (n-12)/2 bytes in size, where n is the integer value stored in the database record header.
H30700: An odd value greater than or equal to 13 stored within the database record header indicates that the corresponding database value is an SQL text field. The blob of data contains the value text stored using the database encoding, with no nul-terminator. The blob of data is exactly (n-12)/2 bytes in size, where n is the integer value stored in the database record header.
H30710: In a well-formed database file, if the values 8 or 9 appear within any database record header within the database, then the schema-layer file format (stored at byte offset 44 of the database file header) must be set to 4.
H30720: In a well-formed database file, the values 10 and 11, and all negative values may not appear within any database record header in the database.
H30730: The pages in an index B-Tree structures are arranged into a tree structure such that all leaf pages are at the same depth.
H30740: Each leaf node page in an index B-Tree contains one or more B-Tree cells, where each cell contains a database record.
H30750: Each internal node page in an index B-Tree contains one or more B-Tree cells, where each cell contains a child page number, C, and a database record R. All database records stored within the sub-tree headed by page C are smaller than record R, according to the index sort order (see below). Additionally, unless R is the smallest database record stored on the internal node page, all integer keys within the sub-tree headed by C are greater than R-1, where R-1 is the largest database record on the internal node page that is smaller than R.
H30760: As well as child page numbers associated with B-Tree cells, each internal node page in an index B-Tree contains the page number of an extra child page, the right-child page. All database records stored in all B-Tree cells within the sub-tree headed by the right-child page are greater than all database records stored within B-Tree cells on the internal node page.
H30770: In a well-formed database, each index B-Tree contains a single entry for each row in the indexed logical database table.
H30780: Each database record (key) stored by an index B-Tree in a well-formed database contains the same number of values, the number of indexed columns plus one.
H30790: The final value in each database record (key) stored by an index B-Tree in a well-formed database contains the rowid (an integer value) of the corresponding logical database row.
H30800: The first N values in each database record (key) stored in an index B-Tree where N is the number of indexed columns, contain the values of the indexed columns from the corresponding logical database row, in the order specified for the index.
H30810: The b-tree page flags field (the first byte) of each database page used as an internal node of an index B-Tree structure is set to 0x02.
H30820: The b-tree page flags field (the first byte) of each database page used as a leaf node of an index B-Tree structure is set to 0x0A.
H30830: The first byte of each database page used as a B-Tree page contains the b-tree page flags field. On page 1, the b-tree page flags field is stored directly after the 100 byte file header at byte offset 100.
H30840: The number of B-Tree cells stored on a B-Tree page is stored as a 2-byte big-endian integer starting at byte offset 3 of the B-Tree page. On page 1, this field is stored at byte offset 103.
H30850: The 2-byte big-endian integer starting at byte offset 5 of each B-Tree page contains the byte-offset from the start of the page to the start of the cell content area, which consumes all space from this offset to the end of the usable region of the page. On page 1, this field is stored at byte offset 105. All B-Tree cells on the page are stored within the cell-content area.
H30860: On each page used as an internal node a of B-Tree structures, the page number of the rightmost child node in the B-Tree structure is stored as a 4-byte big-endian unsigned integer beginning at byte offset 8 of the database page, or byte offset 108 on page 1.
H30870: Immediately following the page header on each B-Tree page is the cell offset array, consisting of N 2-byte big-endian unsigned integers, where N is the number of cells stored on the B-Tree page (H30840). On an internal node B-Tree page, the cell offset array begins at byte offset 12, or on a leaf page, byte offset 8. For the B-Tree node on page 1, these offsets are 112 and 108, respectively.
H30880: The cell offset array and the cell content area (H30850) may not overlap.
H30890: Each value stored in the cell offset array must be greater than or equal to the offset to the cell content area (H30850), and less than the database page size.
H30900: The N values stored within the cell offset array are the byte offsets from the start of the B-Tree page to the beginning of each of the N cells stored on the page.
H30910: No two B-Tree cells may overlap.
H30920: Within the cell content area, all blocks of contiguous free-space (space not used by B-Tree cells) greater than 3 bytes in size are linked together into a linked list, the free block list. Such blocks of free space are known as free blocks.
H30930: The first two bytes of each free block contain the offset of the next free block in the free block list formatted as a 2-byte big-endian integer, relative to the start of the database page. If there is no next free block, then the first two bytes are set to 0x00.
H30940: The second two bytes (byte offsets 2 and 3) of each free block contain the total size of the free block, formatted as a 2-byte big-endian integer.
H30950: On all B-Tree pages, the offset of the first free block in the free block list, relative to the start of the database page, is stored as a 2-byte big-endian integer starting at byte offset 1 of the database page. If there is no first free block (because the free block list is empty), then the two bytes at offsets 1 and 2 of the database page are set to 0x00. On page 1, this field is stored at byte offset 101 of the page.
H30960: Within the cell-content area, all blocks of contiguous free-space (space not used by B-Tree cells) less than or equal to 3 bytes in size are known as fragments. The total size of all fragments on a B-Tree page is stored as a 1-byte unsigned integer at byte offset 7 of the database page. On page 1, this field is stored at byte offset 107.
H30970: Each B-Tree cell belonging to an internal node page of an index B-Tree consists of a 4-byte big-endian unsigned integer, the child page number, followed by a variable length integer field, followed by a database record. The variable length integer field contains the length of the database record in bytes.
H30980: Each B-Tree cell belonging to an leaf page of an index B-Tree consists of a variable length integer field, followed by a database record. The variable length integer field contains the length of the database record in bytes.
H30990:
If the database record stored in an index B-Tree page is
sufficiently small, then the entire cell is stored within the
index B-Tree page. Sufficiently small is defined as equal to or
less than max-local, where:
max-local := (usable-size - 12) * 64 / 255 - 23
H31000: If the database record stored as part of an index B-Tree cell is too large to be stored entirely within the B-Tree page (as defined by H30520), then only a prefix of the database record is stored within the B-Tree page and the remainder stored in an overflow chain. In this case, the database record prefix is immediately followed by the page number of the first page of the overflow chain, formatted as a 4-byte big-endian unsigned integer.
H31010:
When a database record belonging to a table B-Tree cell is
stored partially within an overflow page chain, the size
of the prefix stored within the index B-Tree page is N bytes,
where N is calculated using the following algorithm:
min-local := (usable-size - 12) * 32 / 255 - 23
max-local := (usable-size - 12) * 64 / 255 - 23
N := min-local + ((record-size - min-local) % (usable-size - 4))
if( N > max-local ) N := min-local
H31020: The pages in a table B-Tree structures are arranged into a tree structure such that all leaf pages are at the same depth.
H31030: Each leaf page in a table B-Tree structure contains one or more B-Tree cells, where each cell contains a 64-bit signed integer key value and a database record.
H31040: Each internal node page in a table B-Tree structure contains one or more B-Tree cells, where each cell contains a 64-bit signed integer key value, K, and a child page number, C. All integer key values in all B-Tree cells within the sub-tree headed by page C are less than or equal to K. Additionally, unless K is the smallest integer key value stored on the internal node page, all integer keys within the sub-tree headed by C are greater than K-1, where K-1 is the largest integer key on the internal node page that is smaller than K.
H31050: As well as child page numbers associated with B-Tree cells, each internal node page in a table B-Tree contains the page number of an extra child page, the right-child page. All key values in all B-Tree cells within the sub-tree headed by the right-child page are greater than all key values stored within B-Tree cells on the internal node page.
H31060: In a well-formed database, each table B-Tree contains a single entry for each row in the corresponding logical database table.
H31070: The key value (a 64-bit signed integer) for each B-Tree entry is the same as the value of the rowid field of the corresponding logical database row.
H31080: The SQL values serialized to make up each database record stored as ancillary data in a table B-Tree shall be the equal to the values taken by the N leftmost columns of the corresponding logical database row, where N is the number of values in the database record.
H31090: If a logical database table column is declared as an "INTEGER PRIMARY KEY", then instead of its integer value, an SQL NULL shall be stored in its place in any database records used as ancillary data in a table B-Tree.
H31100: If the database schema layer file-format (the value stored as a 4-byte integer at byte offset 44 of the file header) is 1, then all database records stored as ancillary data in a table B-Tree structure have the same number of fields as there are columns in the corresponding logical database table.
H31110: If the database schema layer file-format value is two or greater and the rightmost M columns of a row contain SQL NULL values, then the corresponding record stored as ancillary data in the table B-Tree has between N-M and N fields, where N is the number of columns in the logical database table.
H31120: If the database schema layer file-format value is three or greater and the rightmost M columns of a row contain their default values according to the logical table declaration, then the corresponding record stored as ancillary data in the table B-Tree may have as few as N-M fields, where N is the number of columns in the logical database table.
H31130: In a well-formed database file, the first byte of each page used as an internal node of a table B-Tree structure is set to 0x05.
H31140: In a well-formed database file, the first byte of each page used as a leaf node of a table B-Tree structure is set to 0x0D.
H31150: B-Tree cells belonging to table B-Tree internal node pages consist of exactly two fields, a 4-byte big-endian unsigned integer immediately followed by a variable length integer. These fields contain the child page number and key value respectively (see H31030).
H31160: B-Tree cells belonging to table B-Tree leaf node pages consist of three fields, two variable length integer values followed by a database record. The size of the database record in bytes is stored in the first of the two variable length integer fields. The second of the two variable length integer fields contains the 64-bit signed integer key (see H31030).
H31170: If the size of the record stored in a table B-Tree leaf page cell is less than or equal to (usable page size-35) bytes, then the entire cell is stored on the B-Tree leaf page. In a well-formed database, usable page size is the same as the database page size.
H31180: If a table B-Tree cell is too large to be stored entirely on a leaf page (as defined by H31170), then a prefix of the cell is stored on the leaf page, and the remainder stored in an overflow page chain. In this case the cell prefix stored on the B-Tree leaf page is immediately followed by a 4-byte big-endian unsigned integer containing the page number of the first overflow page in the chain.
H31190:
When a table B-Tree cell is stored partially in an
overflow page chain, the prefix stored on the B-Tree
leaf page consists of the two variable length integer fields,
followed by the first N bytes of the database record, where
N is determined by the following algorithm:
min-local := (usable-size - 12) * 32 / 255 - 23
max-local := (usable-size - 35)
N := min-local + (record-size - min-local) % (usable-size - 4)
if( N > max-local ) N := min-local
H31200: A single overflow page may store up to available-space bytes of database record data, where available-space is equal to (usable-size - 4).
H31210: When a database record is too large to store within a B-Tree page (see H31170 and H31000), a prefix of the record is stored within the B-Tree page and the remainder stored across N overflow pages. In this case N is the minimum number of pages required to store the portion of the record not stored on the B-Tree page, given the maximum payload per overflow page defined by H31200.
H31220: The list of overflow pages used to store a single database record are linked together in a singly linked list known as an overflow chain. The first four bytes of each page except the last in an overflow chain are used to store the page number of the next page in the linked list, formatted as an unsigned big-endian integer. The first four bytes of the last page in an overflow chain are set to 0x00.
H31230: Each overflow page except the last in an overflow chain contains N bytes of record data starting at byte offset 4 of the page, where N is the maximum payload per overflow page, as defined by H31200. The final page in an overflow chain contains the remaining data, also starting at byte offset 4.
H31240: All free pages in a well-formed database file are part of the database free page list.
H31250: Each free page is either a free list trunk page or a free list leaf page.
H31260: All free list trunk pages are linked together into a singly linked list. The first 4 bytes of each page in the linked list contains the page number of the next page in the list, formatted as an unsigned big-endian integer. The first 4 bytes of the last page in the linked list are set to 0x00.
H31270: The second 4 bytes of each free list trunk page contains the number of free list leaf page numbers stored on the free list trunk page, formatted as an unsigned big-endian integer.
H31280: Beginning at byte offset 8 of each free list trunk page are N page numbers, each formatted as a 4-byte unsigned big-endian integers, where N is the value described in requirement H31270.
H31290: All page numbers stored on all free list trunk pages refer to database pages that are free list leaves.
H31300: The page number of each free list leaf page in a well-formed database file appears exactly once within the set of pages numbers stored on free list trunk pages.
H31310: The total number of pages in the free list, including all free list trunk and free list leaf pages, is stored as a 4-byte unsigned big-endian integer at offset 36 of the database file header.
H31320: The page number of the first page in the linked list of free list trunk pages is stored as a 4-byte big-endian unsigned integer at offset 32 of the database file header. If there are no free list trunk pages in the database file, then the value stored at offset 32 of the database file header is 0.
H31330: Non auto-vacuum databases do not contain pointer map pages.
H31340:
In an auto-vacuum database file, every (num-entries + 1)th
page beginning with page 2 is designated a pointer-map page, where
num-entries is calculated as:
num-entries := database-usable-page-size / 5
H31350: In an auto-vacuum database file, each pointer-map page contains a pointer map entry for each of the num-entries (defined by H31340) pages that follow it, if they exist.
H31360: Each pointer-map page entry consists of a 1-byte page type and a 4-byte page parent number, 5 bytes in total.
H31370: Pointer-map entries are packed into the pointer-map page in order, starting at offset 0. The entry associated with the database page that immediately follows the pointer-map page is located at offset 0. The entry for the following page at offset 5 etc.
H31380: For each page except page 1 in an auto-vacuum database file that is the root page of a B-Tree structure, the page type of the corresponding pointer-map entry is set to the value 0x01 and the parent page number is zero.
H31390: For each page that is a part of an auto-vacuum database file free-list, the page type of the corresponding pointer-map entry is set to the value 0x02 and the parent page number is zero.
H31400: For each page in a well-formed auto-vacuum database that is the first page in an overflow chain, the page type of the corresponding pointer-map entry is set to 0x03 and the parent page number field is set to the page number of the B-Tree page that contains the start of the B-Tree cell stored in the overflow-chain.
H31410: For each page that is the second or a subsequent page in an overflow chain, the page type of the corresponding pointer-map entry is set to 0x04 and the parent page number field is set to the page number of the preceding page in the overflow chain.
H31420: For each page that is not a root page but is a part of a B-Tree tree structure (not part of an overflow chain), the page type of the corresponding pointer-map entry is set to the value 0x05 and the parent page number field is set to the page number of the parent node in the B-Tree structure.
H32000: If a journal file contains a well-formed master-journal pointer and the named master-journal file does not exist then the journal file shall be considered invalid.
H32010: If the first 28 bytes of a journal file do not contain a well-formed journal header, then the journal file shall be considered invalid.
H32020: If the journal file exists within the file-system and neither H32000 , H32010 nor H33080 apply, then the journal file shall be considered valid.
H32030: If there exists a valid journal file in the file-system, then the database page-size in bytes used to interpret the database image shall be the value stored as a 4-byte big-endian unsigned integer at byte offset 24 of the journal file.
H32040: If there exists a valid journal file in the file-system, then the number of pages in the database image shall be the value stored as a 4-byte big-endian unsigned integer at byte offset 24 of the journal file.
H32050: If there is no valid journal file in the file-system, then the database page-size in bytes used to interpret the database image shall be the value stored as a 2-byte big-endian unsigned integer at byte offset 16 of the database file.
H32060: If there is no valid journal file in the file-system, then the number of pages in the database image shall be calculated by dividing the size of the database file in bytes by the database page-size.
H32070: If there exists a valid journal file in the file-system, then the contents of each page of the database image for which there is a valid journal record in the journal file shall be read from the corresponding journal record.
H32080: The contents of all database image pages for which there is no valid journal record shall be read from the database file.
H32090: A buffer of 28 bytes shall be considered a well-formed journal header if it is not excluded by requirements H32180, H32190 or H32200.
H32180: A buffer of 28 bytes shall only be considered a well-formed journal header if the first eight bytes of the buffer contain the values 0xd9, 0xd5, 0x05, 0xf9, 0x20, 0xa1, 0x63, and 0xd7, respectively.
H32190: A buffer of 28 bytes shall only be considered a well-formed journal header if the value stored in the sector size field (the 4-byte big-endian unsigned integer at offset 20 of the buffer) contains a value that is an integer power of two greater than 512.
H32200: A buffer of 28 bytes shall only be considered a well-formed journal header if the value stored in the page size field (the 4-byte big-endian unsigned integer at offset 24 of the buffer) contains a value that is an integer power of two greater than 512.
H32100: A buffer of (8 + page size) bytes shall be considered a well-formed journal record if it is not excluded by requirements H32110 or H32120.
H32110: A journal record shall only be considered to be well-formed if the page number field contains a value other than zero and the locking-page number, calculated using the page size found in the first journal header of the journal file that contains the journal record.
H32120: A journal record shall only be considered to be well-formed if the checksum field contains a value equal to the sum of the value stored in the checksum-initializer field of the journal header that precedes the record and the value stored in every 200th byte of the page data field, interpreted as an 8-bit unsigned integer), starting with byte offset (page-size % 200) and ending with the byte at byte offset (page-size - 200).
H32130: A buffer shall be considered to contain a well-formed master journal pointer record if it is not excluded from this category by requirements H32140, H32150, H32160 or H32170.
H32140: A buffer shall only be considered to be a well-formed master journal pointer if the final eight bytes of the buffer contain the values 0xd9, 0xd5, 0x05, 0xf9, 0x20, 0xa1, 0x63, and 0xd7, respectively.
H32150: A buffer shall only be considered to be a well-formed master journal pointer if the size of the buffer in bytes is equal to the value stored as a 4-byte big-endian unsigned integer starting 16 bytes before the end of the buffer.
H32160: A buffer shall only be considered to be a well-formed master journal pointer if the first four bytes of the buffer, interpreted as a big-endian unsigned integer, contain the page number of the locking page (the value (1 + 230 / page-size), where page-size is the value stored in the page-size field of the first journal header of the journal file).
H32170: A buffer shall only be considered to be a well-formed master journal pointer if the value stored as a 4-byte big-endian integer starting 12 bytes before the end of the buffer is equal to the sum of all bytes, each interpreted as an 8-bit unsigned integer, starting at offset 4 of the buffer and continuing until offset (buffer-size - 16) (the 17th last byte of the buffer).
H32210: A buffer shall be considered to contain a well-formed journal section if it is not excluded from this category by requirements H32220, H32230 or H32240.
H32220: A buffer shall only be considered to contain a well-formed journal section if the first 28 bytes of it contain a well-formed journal header.
H32230: A buffer shall only be considered to contain a well-formed journal section if, beginning at byte offset sector-size, it contains a sequence of record-count well-formed journal records. In this case sector-size and record-count are the integer values stored in the sector size and record count fields of the journal section's journal header.
H32240: A buffer shall only be considered to contain a well-formed journal section if it is an integer multiple of sector-size bytes in size, where sector-size is the value stored in the sector size field of the journal section's journal header.
H32250: A journal record found within a valid journal file shall be considered a valid journal record if it is not excluded from this category by requirement H32260, H32270 or H32280.
H32260: A journal record shall only be considered a valid journal record if it and any other journal records that occur before it within the same journal section are well-formed.
H32270: A journal record shall only be considered a valid journal record if the journal section to which it belongs begins with a well-formed journal header.
H32280: A journal record shall only be considered a valid journal record if all journal sections that occur before the journal section containing the journal record are well-formed journal sections.
H32290: Two database images shall be considered to be equivalent if they (a) have the same page size, (b) contain the same number of pages and (c) the content of each page of the first database image that is not a free-list leaf page is the same as the content of the corresponding page in the second database image.
H32300: If, while writing to an SQLite database file-system representation in order to replace database image A with database image B, a failure that should be handled gracefully occurs, then following recovery the database file-system representation shall contain a database image equivalent to either A or B.
H32320: When using the rollback-journal method to modify the file-system representation of a database image, the database writer shall ensure that before the size of the database file is modified, the first 28 bytes of the journal file contain a stable valid journal header with the page-size and page-count fields set to values corresponding to the original database image.
H32330: When using the rollback-journal method to modify the file-system representation of a database image, the database writer shall ensure that the first 28 bytes of the journal file does not become unstable at any point after the size of the database file is modified until the journal file is invalidated to commit the transaction.
H32340: When using the rollback-journal method to modify the file-system representation of a database image, the database writer shall ensure that before any part of the database file that contained a page of the original database image that was not a free-list leaf page is overwritten or made unstable the journal file contains a valid and stable journal record containing the original page data.
H32350: When using the rollback-journal method to modify the file-system representation of a database image, the database writer shall ensure that after any part of the database file that contained a page of the original database image that was not a free-list leaf page has been overwritten or made unstable the corresponding journal record (see H32340) is not modified or made unstable.
H32360: When using the rollback-journal method to modify the file-system representation of a database image, the database writer shall ensure that before the database file is truncated, the journal file contains stable valid journal records corresponding to all pages of the original database image that were part of the region being discarded by the truncate operation and were not free-list leaf pages.
H32370: When using the rollback-journal method to modify the file-system representation of a database image, the database writer shall ensure that after the database file has been truncated the journal records corresponding to pages from the original database image that were part of the truncated region and were not free-list leaf pages are not modified or made unstable.
H33000: Before reading from a database file , a database reader shall establish a SHARED or greater lock on the database file-system representation.
H33010: Before writing to a database file, a database writer shall establish an EXCLUSIVE lock on the database file-system representation.
H33020: Before writing to a journal file, a database writer shall establish a RESERVED, PENDING or EXCLUSIVE lock on the database file-system representation.
H33030: Before establishing a RESERVED or PENDING lock on a database file, a database writer shall ensure that the database file contains a valid database image.
H33060: Before establishing a RESERVED or PENDING lock on a database file, a database writer shall ensure that any journal file that may be present is not a valid journal file.
H33080: If another database client holds either a RESERVED or PENDING lock on the database file-system representation, then any journal file that exists within the file system shall be considered invalid.
H33040: A database writer shall increment the value of the database header change counter field, a 4-byte big-endian unsigned integer field stored at byte offset 24 of the database header, as part of the first database image modification that it performs after obtaining an EXCLUSIVE lock.
H33050: A database writer shall increment the value of the database schema version field, a 4-byte big-endian unsigned integer field stored at byte offset 40 of the database header, as part of the first database image modification that includes a schema change that it performs after obtaining an EXCLUSIVE lock.
H33070: If a database writer is required by either H33050 or H33040 to increment a database header field, and that header field already contains the maximum value possible (0xFFFFFFFF, or 4294967295 for 32-bit unsigned integer fields), "incrementing" the field shall be interpreted to mean setting it to zero.
H35010: Except for the read operation required by H35070 and those reads made as part of opening a read-only transaction, SQLite shall ensure that a database connection has an open read-only or read/write transaction when any data is read from the database file.
H35020: Aside from those read operations described by H35070 and H21XXX, SQLite shall read data from the database file in aligned blocks of page-size bytes, where page-size is the database page size used by the database file.
H35030: While opening a read-only transaction, after successfully obtaining a shared lock on the database file, SQLite shall attempt to detect and roll back a hot journal file associated with the same database file.
H35040: Assuming no errors have occured, then after attempting to detect and roll back a hot journal file, if the page cache contains any entries associated with the current database connection, then SQLite shall validate the contents of the page cache by testing the file change counter. This procedure is known as cache validiation.
H35050: If the cache validiate procedure prescribed by H35040 is required and does not prove that the page cache entries associated with the current database connection are valid, then SQLite shall discard all entries associated with the current database connection from the page cache.
H35060: When a new database connection is required, SQLite shall attempt to open a file-handle on the database file. If the attempt fails, then no new database connection is created and an error returned.
H35070: When a new database connection is required, after opening the new file-handle, SQLite shall attempt to read the first 100 bytes of the database file. If the attempt fails for any other reason than that the opened file is less than 100 bytes in size, then the file-handle is closed, no new database connection is created and an error returned instead.
H35080: If the database file header is successfully read from a newly opened database file, the connections expected page-size shall be set to the value stored in the page-size field of the database header.
H35090: If the database file header cannot be read from a newly opened database file (because the file is less than 100 bytes in size), the connections expected page-size shall be set to the compile time value of the SQLITE_DEFAULT_PAGESIZE option.
H35100: When required to open a read-only transaction using a database connection, SQLite shall first attempt to obtain a shared-lock on the file-handle open on the database file.
H35110: If, while opening a read-only transaction, SQLite fails to obtain the shared-lock on the database file, then the process is abandoned, no transaction is opened and an error returned to the user.
H35120: If, while opening a read-only transaction, SQLite encounters an error while attempting to detect or roll back a hot journal file, then the shared-lock on the database file is released, no transaction is opened and an error returned to the user.
H35130: When required to end a read-only transaction, SQLite shall relinquish the shared lock held on the database file by calling the xUnlock() method of the file-handle.
H35140: When required to attempt to detect a hot-journal file, SQLite shall first use the xAccess() method of the VFS layer to check if a journal file exists in the file-system.
H35150: When required to attempt to detect a hot-journal file, if the call to xAccess() required by H35140 indicates that a journal file does not exist, then SQLite shall conclude that there is no hot-journal file in the file system and therefore that no hot journal rollback is required.
H35160: When required to attempt to detect a hot-journal file, if the call to xAccess() required by H35140 indicates that a journal file is present, then the xCheckReservedLock() method of the database file file-handle is invoked to determine whether or not some other process is holding a reserved or greater lock on the database file.
H35170: If the call to xCheckReservedLock() required by H35160 indicates that some other database connection is holding a reserved or greater lock on the database file, then SQLite shall conclude that there is no hot journal file. In this case the attempt to detect a hot journal file is concluded.
H35180: When a file-handle open on a database file is unlocked, if the page cache contains one or more entries belonging to the associated database connection, SQLite shall store the value of the file change counter internally.
H35190: When required to perform cache validation as part of opening a read transaction, SQLite shall read a 16 byte block starting at byte offset 24 of the database file using the xRead() method of the database connections file handle.
H35200: While performing cache validation, after loading the 16 byte block as required by H35190, SQLite shall compare the 32-bit big-endian integer stored in the first 4 bytes of the block to the most recently stored value of the file change counter (see H35180). If the values are not the same, then SQLite shall conclude that the contents of the cache are invalid.
H35210: During the conclusion of a read transaction, before unlocking the database file, SQLite shall set the connections expected page size to the current database page-size.
H35220: As part of opening a new read transaction, immediately after performing cache validation, if there is no data for database page 1 in the page cache, SQLite shall read N bytes from the start of the database file using the xRead() method of the connections file handle, where N is the connections current expected page size value.
H35230: If page 1 data is read as required by H35230, then the value of the page-size field that appears in the database file header that consumes the first 100 bytes of the read block is not the same as the connections current expected page size, then the expected page size is set to this value, the database file is unlocked and the entire procedure to open a read transaction is repeated.
H35240: If page 1 data is read as required by H35230, then the value of the page-size field that appears in the database file header that consumes the first 100 bytes of the read block is the same as the connections current expected page size, then the block of data read is stored in the page cache as page 1.
H35270: When required to journal a database page, SQLite shall first append the page number of the page being journalled to the journal file, formatted as a 4-byte big-endian unsigned integer, using a single call to the xWrite method of the file-handle opened on the journal file.
H35280: When required to journal a database page, if the attempt to append the page number to the journal file is successful, then the current page data (page-size bytes) shall be appended to the journal file, using a single call to the xWrite method of the file-handle opened on the journal file.
H35290: When required to journal a database page, if the attempt to append the current page data to the journal file is successful, then SQLite shall append a 4-byte big-endian integer checksum value to the to the journal file, using a single call to the xWrite method of the file-handle opened on the journal file.
H35300: The checksum value written to the journal file by the write required by H35290 shall be equal to the sum of the checksum initializer field stored in the journal header (H35700) and every 200th byte of the page data, beginning with the (page-size % 200)th byte.
H35350: When required to open a write transaction on the database, SQLite shall first open a read transaction, if the database connection in question has not already opened one.
H35360: When required to open a write transaction on the database, after ensuring a read transaction has already been opened, SQLite shall obtain a reserved lock on the database file by calling the xLock method of the file-handle open on the database file.
H35370: When required to open a write transaction on the database, after obtaining a reserved lock on the database file, SQLite shall open a read/write file-handle on the corresponding journal file.
H35380: When required to open a write transaction on the database, after opening a file-handle on the journal file, SQLite shall append a journal header to the (currently empty) journal file.
H35400: When a database connection is closed, SQLite shall close the associated file handle at the VFS level.
H35420: SQLite shall ensure that a database connection has an open read-only or read/write transaction before using data stored in the page cache to satisfy user queries.
H35430: When a database connection is closed, all associated page cache entries shall be discarded.
H35440: If while attempting to detect a hot-journal file the call to xCheckReservedLock() indicates that no process holds a reserved or greater lock on the database file, then SQLite shall open a file handle on the potentially hot journal file using the VFS xOpen() method.
H35450: After successfully opening a file-handle on a potentially hot journal file, SQLite shall query the file for its size in bytes using the xFileSize() method of the open file handle.
H35460: If the size of a potentially hot journal file is revealed to be zero bytes by a query required by H35450, then SQLite shall close the file handle opened on the journal file and delete the journal file using a call to the VFS xDelete() method. In this case SQLite shall conclude that there is no hot journal file.
H35470: If the size of a potentially hot journal file is revealed to be greater than zero bytes by a query required by H35450, then SQLite shall attempt to upgrade the shared lock held by the database connection on the database file directly to an exclusive lock.
H35480: If an attempt to upgrade to an exclusive lock prescribed by H35470 fails for any reason, then SQLite shall release all locks held by the database connection and close the file handle opened on the journal file. The attempt to open a read-only transaction shall be deemed to have failed and an error returned to the user.
H35490: If, as part of the hot journal file detection process, the attempt to upgrade to an exclusive lock mandated by H35470 is successful, then SQLite shall query the file-system using the xAccess() method of the VFS implementation to test whether or not the journal file is still present in the file-system.
H35500: If the xAccess() query required by H35490 reveals that the journal file is still present in the file system, then SQLite shall conclude that the journal file is a hot journal file that needs to be rolled back. SQLite shall immediately begin hot journal rollback.
H35510: If the call to xAccess() required by H35140 fails (due to an IO error or similar), then SQLite shall abandon the attempt to open a read-only transaction, relinquish the shared lock held on the database file and return an error to the user.
H35520: If the call to xCheckReservedLock() required by H35160 fails (due to an IO or other internal VFS error), then SQLite shall abandon the attempt to open a read-only transaction, relinquish the shared lock held on the database file and return an error to the user.
H35530: If the call to xOpen() required by H35440 fails (due to an IO or other internal VFS error), then SQLite shall abandon the attempt to open a read-only transaction, relinquish the shared lock held on the database file and return an error to the user.
H35540: If the call to xFileSize() required by H35450 fails (due to an IO or other internal VFS error), then SQLite shall abandon the attempt to open a read-only transaction, relinquish the shared lock held on the database file, close the file handle opened on the journal file and return an error to the user.
H35550: If the call to xDelete() required by H35450 fails (due to an IO or other internal VFS error), then SQLite shall abandon the attempt to open a read-only transaction, relinquish the shared lock held on the database file and return an error to the user.
H35560: If the call to xAccess() required by H35490 fails (due to an IO or other internal VFS error), then SQLite shall abandon the attempt to open a read-only transaction, relinquish the lock held on the database file, close the file handle opened on the journal file and return an error to the user.
H35570: If the call to xAccess() required by H35490 reveals that the journal file is no longer present in the file system, then SQLite shall abandon the attempt to open a read-only transaction, relinquish the lock held on the database file, close the file handle opened on the journal file and return an SQLITE_BUSY error to the user.
H35580: If an attempt to acquire a reserved lock prescribed by requirement H35360 fails, then SQLite shall deem the attempt to open a write transaction to have failed and return an error to the user.
H35590: When required to modify the contents of an existing database page that existed and was not a free-list leaf page when the write transaction was opened, SQLite shall journal the page if it has not already been journalled within the current write transaction.
H35600: When required to modify the contents of an existing database page, SQLite shall update the cached version of the database page content stored as part of the page cache entry associated with the page.
H35610: When required to append a new database page to the database file, SQLite shall create a new page cache entry corresponding to the new page and insert it into the page cache. The dirty flag of the new page cache entry shall be set.
H35620: When required to truncate (remove) a database page that existed and was not a free-list leaf page when the write transaction was opened from the end of a database file, SQLite shall journal the page if it has not already been journalled within the current write transaction.
H35630: When required to truncate a database page from the end of the database file, SQLite shall discard the associated page cache entry from the page cache.
H35640: When required to purge a non-writable dirty page from the page cache, SQLite shall sync the journal file before proceding with the write operation required by H35670.
H35660: After syncing the journal file as required by H35640, SQLite shall append a new journal header to the journal file before proceding with the write operation required by H35670.
H35670: When required to purge a page cache entry that is a dirty page SQLite shall write the page data into the database file, using a single call to the xWrite method of the database connection file handle.
H35680: When required to append a journal header to the journal file, SQLite shall do so by writing a block of sector-size bytes using a single call to the xWrite method of the file-handle open on the journal file. The block of data written shall begin at the smallest sector-size aligned offset at or following the current end of the journal file.
H35690: The first 8 bytes of the journal header required to be written by H35680 shall contain the following values, in order from byte offset 0 to 7: 0xd9, 0xd5, 0x05, 0xf9, 0x20, 0xa1, 0x63 and 0xd7.
H35700: Bytes 8-11 of the journal header required to be written by H35680 shall contain 0x00.
H35710: Bytes 12-15 of the journal header required to be written by H35680 shall contain the number of pages that the database file contained when the current write-transaction was started, formatted as a 4-byte big-endian unsigned integer.
H35720: Bytes 16-19 of the journal header required to be written by H35680 shall contain pseudo-randomly generated values.
H35730: Bytes 20-23 of the journal header required to be written by H35680 shall contain the sector size used by the VFS layer, formatted as a 4-byte big-endian unsigned integer.
H35740: Bytes 24-27 of the journal header required to be written by H35680 shall contain the page size used by the database at the start of the write transaction, formatted as a 4-byte big-endian unsigned integer.
H35750: When required to sync the journal file, SQLite shall invoke the xSync method of the file handle open on the journal file.
H35760: When required to sync the journal file, after invoking the xSync method as required by H35750, SQLite shall update the record count of the journal header most recently written to the journal file. The 4-byte field shall be updated to contain the number of journal records that have been written to the journal file since the journal header was written, formatted as a 4-byte big-endian unsigned integer.
H35770: When required to sync the journal file, after updating the record count field of a journal header as required by H35760, SQLite shall invoke the xSync method of the file handle open on the journal file.
H35780: When required to upgrade to an exclusive lock as part of a write transaction, SQLite shall first attempt to obtain a pending lock on the database file if one is not already held by invoking the xLock method of the file handle opened on the database file.
H35790: When required to upgrade to an exclusive lock as part of a write transaction, after successfully obtaining a pending lock SQLite shall attempt to obtain an exclusive lock by invoking the xLock method of the file handle opened on the database file.
H35800: When required to commit a write-transaction, SQLite shall modify page 1 to increment the value stored in the change counter field of the database file header.
H35810: When required to commit a write-transaction, after incrementing the change counter field, SQLite shall sync the journal file.
H35820: When required to commit a write-transaction, after syncing the journal file as required by H35810, if an exclusive lock on the database file is not already held, SQLite shall attempt to upgrade to an exclusive lock.
H35830: When required to commit a write-transaction, after syncing the journal file as required by H35810 and ensuring that an exclusive lock is held on the database file as required by H35830, SQLite shall copy the contents of all dirty page stored in the page cache into the database file using calls to the xWrite method of the database connection file handle. Each call to xWrite shall write the contents of a single dirty page (page-size bytes of data) to the database file. Dirty pages shall be written in order of page number, from lowest to highest.
H35840: When required to commit a write-transaction, after copying the contents of any dirty pages to the database file as required by H35830, SQLite shall sync the database file by invoking the xSync method of the database connection file handle.
H35850: When required to commit a write-transaction, after syncing the database file as required by H35840, SQLite shall close the file-handle opened on the journal file and delete the journal file from the file system via a call to the VFS xDelete method.
H35860: When required to commit a write-transaction, after deleting the journal file as required by H35850, SQLite shall relinquish all locks held on the database file by invoking the xUnlock method of the database connection file handle.
*** DRAFT ***