SQLite User Forum

Order of the column values on a leaf page
Login

Order of the column values on a leaf page

(1) By Darick Tong (darick) on 2025-04-29 20:28:44 [link] [source]

The cid column returned for PRAGMA table_info is described as indicating the "rank within the current result set".

Is this equivalent to the ordering in which the associated values appear in the Record portion of a B-tree table leaf page?

If not, is there a more official way of determining the latter?

Thank you!

(2) By Gunter Hick (gunter_hick) on 2025-04-30 06:07:50 in reply to 1 [link] [source]

No.

The cid column is a zero-based numbering of the output rows of the pragma that indicates the order in which the rows are output from the query that implements the pragma. It is NOT an attribute of the field (=row) itself, nor does it necessarily reflect the order in which the fields appeared in the CREATE TABLE statement or the order in which the fields are stored within a record. It MAY be so sometimes (or even mostly), but SQLite docs expressly exclude any guarantee thereof.

You cannot determine the order of table records within a B-tree table leaf page. You cannot even determine if two given records are in the same B-tree table leaf page. This should already be apparent from the description of the file format. https://sqlite.org/fileformat2.html

If all you ever do to a table (that was created without an explicit alias for the internal rowid) is INSERT and SELECT, then table records will tend to be stored in order of rowid. The greater the difference between rowids, the more likely is is that there will be at least one "page break" between them.

As soon as you start UPDATEing (which could change their size, depending on the values of the fields) or DELETEing (which creates "holes" in the leaf pages, that may or may not be reused) rows, SQLite may need to move records within (or even between) leaf pages, which may lead to out-of-order storage of rows.

Apart from curiosity, there is no reason to be concerned with the internal storage order of a conceptually unordered set, aka table. Indeed it is highly dangerous to rely on any implicit ordering, as this order may change without warning, which is the reason one is required to provide an ORDER BY clause if one desires a specfic order in which result rows are to be returned.

(3) By Darick Tong (darick) on 2025-04-30 22:18:31 in reply to 2 [link] [source]

Thank you Gunter!

Just to be clear, I am not interested in or referring to the order of rows / records in the B-tree table leaf page. Rather, I am interested in the order of the column values within a row/record.

I see that you mention that "[the cid column] does not necessarily reflect ... the order in which the fields are stored within a record". May I ask how that ordering is determined? Happy to follow pointers to the code too if that's easier!

(4) By Adrian Ho (lexfiend) on 2025-04-30 23:29:09 in reply to 3 [link] [source]

May I ask how that ordering is determined?

As The Fine Docs state:

The order of values in the record is the same as the order of columns in the SQL table definition.

(5) By Darick Tong (darick) on 2025-05-01 00:26:48 in reply to 4 [link] [source]

Ah, there it is, thank you Adrian.

Last question:

Is there a way to determine that column order, outside of parsing the SQL table definition?

It sounds like the cid value will sometimes match that order but it is not guaranteed to. Is there a different place where that position is stored in the db file, or is it strictly in-memory state that SQLite initializes when reading the database (which would require parsing the CREATE statements on every connect)?

(6) By Bo Lindbergh (_blgl_) on 2025-05-01 00:39:38 in reply to 5 [link] [source]

So you're trying to read an SQLite database file without using the SQLite library, but you want to use the SQLite library to run pragmas? That's cheating....

(7) By Adrian Ho (lexfiend) on 2025-05-01 04:55:40 in reply to 6 [link] [source]

I'd say that's unwise myself. Darick, what problem are you trying to solve with "mixed mode" DB access?

(8) By Darick Tong (darick) on 2025-05-03 01:31:12 in reply to 7 [source]

I'm using litestream to incrementally backup the database, and am looking into adding visibility of the progress of the backup (outside of restoring the entire database and querying it).

The current approach is with a fork of litestream that watches/exports the value of column in a single-row table (that tracks the database version), based on the pages that it successfully backs up.