SQLite User Forum

Patches to compile SQLite with non-Unicode
Login

Patches to compile SQLite with non-Unicode

(1) By anonymous on 2022-08-03 22:46:30 [link] [source]

I wrote a patch to compile SQLite with non-Unicode capabilities, which is public domain. (There are some other patches that I wanted to apply too, but this file does not include them. In some cases, it cannot include them, for reasons explained in the comments.) I wrote also the program which will apply the patch.

The hash.c and hash.h files can be found in the Free Hero Mesh repository (and some parts of them are based on the public domain code in SQLite, too). (Later, these patches might also be applied to Free Hero Mesh, too, but currently it is not.)

Please report if some things do not work properly (including if some version of SQLite breaks it), or other comments/complaints. There is also possibility that some further enhancements would be possible including ones that would improve performance in other ways than what I had done.

(2) By anonymous on 2022-08-04 17:08:40 in reply to 1 [link] [source]

Is there list of patches and extensions of SQLite, that can be made up?

Are there suggestions of dealing with the things that I have mentioned cannot effectively be patched (and anything else that might be not effectively able to be patched)? (Also, the SQL syntax cannot effectively be patched if working with the amalgamation; that would require working with the separated source files, I think.)

I have seen other patches too, such as patches for strftime function, etc. Hopefully a collection of patches will be possible.

Would any such things later be added into SQLite?

(3) By Richard Hipp (drh) on 2022-08-04 18:06:09 in reply to 2 [link] [source]

Is there list of patches and extensions of SQLite

I am not aware of any such list. You are welcomed to start one, if you like.

You are also welcomed to make and maintain your own patches, or your own alternative SQLite source tree. I see two relatively straight-forward ways of doing this:

  1. Clone the Fossil repository for SQLite located at https://sqlite.org/src and stand up your clone on your own website. Keep your patches in a branch, and keep that branch up to date with trunk by periodically merging in changes from trunk.

  2. Fork the SQLite GitHub mirror and make your patches in the fork. Again, you will need to keep your fork up to date by periodically merging in changes from master.

If what you are really trying to ask is whether we will take your patches and maintain them for you for the next 30 years for free, then the answer is probably "No".

(4) By anonymous on 2022-08-04 20:07:47 in reply to 3 [link] [source]

I am not aware of any such list. You are welcomed to start one, if you like.

OK. There is this patch, and there is also the strftime patch (which has been posted recently in this forum, too), and probably others that I do not know about.

I will start to make a list (and then post a message once I have done so), and hopefully someone might have comment of it too, which might be needed. Maybe I will make a Fossil repository with a list of the patches (and I can include extensions too). Also useful to have might be to have the list to mention if some patches (and/or compile-time options) conflict, etc. And then, test cases for patches, etc.

Even in this specific case (the non-Unicode patch), there are possible enhancements, e.g. specifying in the database header, and I think there is a LENGTH optimization which now could be used with TEXT as well as BLOB, and the (noted in the comments of the patch file) working with Windows file names (a URI parameter could be used to select the encoding, I suppose), etc. (I do not use Windows, so I am not changing any of the Windows-related code, but someone else who wants this feature on Windows might contribute it.) (I also think that the URI format might not have been the best idea to handle these kind of parameters, although now it is there. I could add into the patch, another function which accepts an array of parameters instead, I suppose; but for now I did not do it.)

I can then add additional commands into the patching program if necessary for some kind of patches, too.

You are also welcomed to make and maintain your own patches, or your own alternative SQLite source tree. I see two relatively straight-forward ways of doing this

A third way is to patch the amalgamation like I had done, which seems to be easier to work with in common cases, although some kind of patches are not suitable for using this method of doing so, such as those involving changing the parser.

If what you are really trying to ask is whether we will take your patches and maintain them for you for the next 30 years for free, then the answer is probably "No".

That wasn't my first question, but my last one is similar but not quite like that. Rather, what I was asking is some of the functionality of any of these (and/or any other) patches might later be made official capabilities of SQLite.

(I have seen apparently some older versions of SQLite before version 3 does have the ability to be compiled for Unicode or non-Unicode mode, although I do not know the details, or how similar it is to my patches.)

In some cases, some features that might want to be added by patches would require adding new numbers into enumerations and fields into public structures, although I have the reason why I do not want to do that in unofficial patches while SQLite is still officially being maintained; they are too likely to conflict with official features of SQLite in future. This is even though there are some features that I had wanted, which that would seems best way to do it, though.

(5) By Donal Fellows (dkfellows) on 2022-08-05 13:05:40 in reply to 4 [link] [source]

In some cases, some features that might want to be added by patches would require adding new numbers into enumerations and fields into public structures, although I have the reason why I do not want to do that in unofficial patches while SQLite is still officially being maintained; they are too likely to conflict with official features of SQLite in future.

If the enum in question isn't modelling a bit field, just picking actual values much larger than anything that exists is likely to be the best strategy. Bitfields are much harder to extend as they're a more precious resource (typically just 31 values are usable in a moderately portable way, as opposed to 2³¹ values — neither can safely use the top bit because of the fun of signed ints).

It's hard to discuss generalities in this area. Specific suggestions are easier.

(6) By anonymous on 2022-08-05 18:56:51 in reply to 5 [source]

Some cases are:

  • C functions that would need to be callable by loadable extensions (the sqlite3_api_routines structure). (One rather messy way to do it would be a SQL function that accepts a pointer to a variable to which to write a pointer to a structure for the extra unofficial C functions.)

  • File control opcodes, virtual table configuration codes. In this case it is an ordinary enumeration, so big numbers could be used to try to avoid conflicts, I suppose. (In some cases, it might be possible to enhance the patching program so that commands can be added to detect conflicts, whether with official codes or with other patches.)

  • Additional flags for sqlite3_create_function and its variants, and for sqlite3_prepare_v3. These do use bit fields so it is more difficult.

  • Adding new methods into structures such as virtual tables, VFS, etc. In this case there is a version number (which cannot easily be assigned without breaking future compatibility) and extra fields added to the structure (which can conflict, especially for loadable extensions). (You could use sqlite3_compileoption_used to detect the presence of patches which use this mechanism to expose their presence (which is what I have done in the non-Unicode patch), but you might want virtual table modules to be .)

  • Adding new SQL syntax. In this case, the patches cannot (easily) be made to work with the amalgamation; instead it will be necessary to work with the full source codes of SQLite.

  • Any internal codes which might conflict if multiple patches are applied. In this case, if I keep a collection of such patches then I could try to change them so that they will not conflict, or anyone else who adds any can do so, too.

For these and other reasons, adding unofficial patches to add the desired features is not as good as implementing them officially, although in some cases they could be used.

I also have some specific examples of things that I might want to be added, such as: SQLITE_PREPARE_MULTI (to prepare multiple SQL statements that will be executed in sequence, but with some restrictions about which combination of statements are possible; you can then also assign their parameters all together, too). I also have some ideas about virtual tables, but am unsure the best way that they should be done. (In some cases, they have already been added officially into SQLite (sometimes independently from my suggestions; other people may sometimes have similar ideas), but sometimes not.)

And then there is also patching the shell, which I had not done, but may also be desirable.