SQLite Forum

Subclassing & private data
Login

Subclassing & private data

(1) By Banana on 2021-10-01 21:20:44 [link] [source]

From this page, we have this paragraph:

Virtual table implementations will normally subclass this structure to add additional private and implementation-specific fields.

and

Once again, practical implementations will likely subclass this structure to add additional private fields.

Because I only know enough to shoot my feet in C, I'm wondering how that is going to work out.

Looking at the samples, I see the typical pattern is to make the struct the first field of the actual struct, e.g.:

typedef struct JsonEachCursor JsonEachCursor;
struct JsonEachCursor {
  sqlite3_vtab_cursor base;  /* Base class - must be first */
  u32 iRowid;                /* The rowid */
  u32 iBegin;                /* The first node of the scan */
  u32 i;                     /* Index in sParse.aNode[] of current row */
  u32 iEnd;                  /* EOF when i equals or exceeds this value */
  u8 eType;                  /* Type of top-level element */
  u8 bRecursive;             /* True for json_tree().  False for json_each() */
  char *zJson;               /* Input JSON */
  char *zRoot;               /* Path by which to filter zJson */
  JsonParse sParse;          /* Parse of the input JSON */
};

However, I wonder how versioning is supposed to work. Some implementation will put in a size parameter to size the structure as to infer the version of the struct. Without that information, a new version of struct could potentially read private data from the old version and therefore garbage. But I don't see such member in sqlite's structure. Does the member iVersion of the struct sqlite3_module handle this? This statement implies it only applies to the module struct, not to the virtual table or cursor struct:

The module structure also contains the iVersion field which defines the particular edition of the module table structure. Currently, iVersion is always 3 or less, but in future releases of SQLite the module structure definition might be extended with additional methods and in that case the maximum iVersion value will be increased.

I wanted to be sure that I'm following all the conventions that is expected as C seems to rely more on the conventions for it to work as opposed to other languages which may enforce the shape via its compiler. Thanks in advance!

(2) By Larry Brasfield (larrybr) on 2021-10-01 22:33:58 in reply to 1 [link] [source]

However, I wonder how versioning is supposed to work. Some implementation will put in a size parameter to size the structure as to infer the version of the struct. Without that information, a new version of struct could potentially read private data from the old version and therefore garbage. But I don't see such member in sqlite's structure. Does the member iVersion of the struct sqlite3_module handle this?

I'm reasonably sure that the iVersion member is intended to deal with attempts to use extensions written for SQLite major version 3 with some later version(s). In other words, it addresses possible changes in the extension host.

The problem you are wondering about is how to deal with changes in your own code. (I presume you are writing such, or contemplate doing so.) For that, you probably need a mechanism that is available even before attempting to "attach" an extension after it is loaded. And that mechanism will be one you devise or model after similar solutions. This is a non-trivial problem, especially with dynamically loaded code. And the solutions vary according to how much and what kind of change is anticipated. You should not expect a ready-made solution to that general problem in the SQLite library or project.

I say this to perhaps put an end to your wondering how that problem is already solved. It is not.

You may recognize, in the pre-defined structs for various extension types, the polymorphism that is available in C++ and other fully OO languages. Typically, that is utilized purely through variation in what the methods do, without relying on varying data being somehow dealt with at the interface. I suggest that this sort of discipline will simplify how you might insulate your extension design from implementation changes or behavior evolution at the interface.

(3) By Banana on 2021-10-02 02:17:27 in reply to 2 [source]

The problem you are wondering about is how to deal with changes in your own code.

Actually, no. I'm wondering how do I write my code as to be forward compatible with future versions of Sqlite that may add new members to the structs without breaking my code written for the current versions.

As I mentioned in my OP, C is not my forte and from what I saw so far, C has the convention of having a field in a struct to report the size of the structure or in Sqlite's case, the version supported. What I was uncertain is that the field doesn't exist on all structs (in this case, the sqlite3_vtab, sqlite3_vtab_cursor, and sqlite3_index_info). That seems to go against what I thought was the convention in handling versions of struct in C.

The linked documentations indicates an expectation that the sqlite3_vtab and sqlite3_vtab_cursor will be likely subclassed (and we can see that in the sample), but those don't have a field that reports versions or size of the structure. How will I know that I won't be writing code that will break in a future version of Sqlite?

It might be that I'm not fully understanding the conventions used here and I want to make explicit the implicit convention that may be used here. I believe that is the first time I saw a version field being used for more than one structs when in other uses, every struct always has a field to report the version.

(4) By Larry Brasfield (larrybr) on 2021-10-02 02:37:47 in reply to 3 [link] [source]

Actually, no. I'm wondering how do I write my code as to be forward compatible with future versions of Sqlite that may add new members to the structs without breaking my code written for the current versions.

If the SQLite library were to change the definition of the leading (or not yet "subclassed") portion of the structs it uses at the extension interfaces, it would break a great many extensions compiled against earlier versions of that publicly exposed data. This kind of backward incompatibility is carefully avoided by the project's management and the developers generally.

As I mentioned in my OP, C is not my forte and from what I saw so far, C has the convention of having a field in a struct to report the size of the structure or in Sqlite's case, the version supported.

Neither of those common practices is a C convention. You accurately perceive what the practice accomplishes. If the library designer(s) had elected to leave room for later, changed versions of those structs, a member named something like sizeof_public_portion might well have been used. However, along with such changed data structure would come changed requirements as to what is done with it or perhaps changed allowance as to what the extension might do with it. Those intriguing options for change are closed off by the API stability guarantees that are part of the library's interface.

How will I know that I won't be writing code that will break in a future version of Sqlite?

The same way that you know sqlite3_open() will do the same thing next decade as it does today.