SQLite Forum

Timeline
Login

5 forum posts by user Banana

2021-10-02
14:02 Reply: MicrosoftODBC Driver Manager Data source name not found and no default driver specified error when use VBA connect Sqlite (artifact: b03364c533 user: Banana)

Actually, this is not completely accurate. There is 64-bit ADO, otherwise a lot of Office documents depending on ADO would break on 64 bit Office.

The problem is likely with trying to use provider MSDASQL (aka ODBC over OLEDB provider) which is implicit in the connection (the provider keyword wasn’t specified, and the ODBC connection syntax is used rather than OLEDB connection syntax. This causes ADO to use MSDASQL as the default provider).

I suggest that the OP start with this SO thread and continue discussion there or start a new SO thread.

02:17 Reply: Subclassing & private data (artifact: af0e394657 user: Banana)

The problem you are wondering about is how to deal with changes in your own code.

Actually, no. I'm wondering how do I write my code as to be forward compatible with future versions of Sqlite that may add new members to the structs without breaking my code written for the current versions.

As I mentioned in my OP, C is not my forte and from what I saw so far, C has the convention of having a field in a struct to report the size of the structure or in Sqlite's case, the version supported. What I was uncertain is that the field doesn't exist on all structs (in this case, the sqlite3_vtab, sqlite3_vtab_cursor, and sqlite3_index_info). That seems to go against what I thought was the convention in handling versions of struct in C.

The linked documentations indicates an expectation that the sqlite3_vtab and sqlite3_vtab_cursor will be likely subclassed (and we can see that in the sample), but those don't have a field that reports versions or size of the structure. How will I know that I won't be writing code that will break in a future version of Sqlite?

It might be that I'm not fully understanding the conventions used here and I want to make explicit the implicit convention that may be used here. I believe that is the first time I saw a version field being used for more than one structs when in other uses, every struct always has a field to report the version.

2021-10-01
21:20 Post: Subclassing & private data (artifact: a724f67185 user: Banana)

From this page, we have this paragraph:

Virtual table implementations will normally subclass this structure to add additional private and implementation-specific fields.

and

Once again, practical implementations will likely subclass this structure to add additional private fields.

Because I only know enough to shoot my feet in C, I'm wondering how that is going to work out.

Looking at the samples, I see the typical pattern is to make the struct the first field of the actual struct, e.g.:

typedef struct JsonEachCursor JsonEachCursor;
struct JsonEachCursor {
  sqlite3_vtab_cursor base;  /* Base class - must be first */
  u32 iRowid;                /* The rowid */
  u32 iBegin;                /* The first node of the scan */
  u32 i;                     /* Index in sParse.aNode[] of current row */
  u32 iEnd;                  /* EOF when i equals or exceeds this value */
  u8 eType;                  /* Type of top-level element */
  u8 bRecursive;             /* True for json_tree().  False for json_each() */
  char *zJson;               /* Input JSON */
  char *zRoot;               /* Path by which to filter zJson */
  JsonParse sParse;          /* Parse of the input JSON */
};

However, I wonder how versioning is supposed to work. Some implementation will put in a size parameter to size the structure as to infer the version of the struct. Without that information, a new version of struct could potentially read private data from the old version and therefore garbage. But I don't see such member in sqlite's structure. Does the member iVersion of the struct sqlite3_module handle this? This statement implies it only applies to the module struct, not to the virtual table or cursor struct:

The module structure also contains the iVersion field which defines the particular edition of the module table structure. Currently, iVersion is always 3 or less, but in future releases of SQLite the module structure definition might be extended with additional methods and in that case the maximum iVersion value will be increased.

I wanted to be sure that I'm following all the conventions that is expected as C seems to rely more on the conventions for it to work as opposed to other languages which may enforce the shape via its compiler. Thanks in advance!

2021-09-28
16:38 Edit: Compiling sqlite3 on Windows with ICU enabled (artifact: 2e66ee3227 user: Banana)

I've found that the instructions seems to be missing some crucial pieces of data and there may be some errors in the MakeFile.msc when one wants to compile the sqlite3 with ICU options enabled. There are also difference references on using ICU.

  1. The instructions fails to mention to set up the Tcl. I had made the mistake of skipping over the section #3 and going directly to section #5, thinking previous sections were POSIX-specific and not applicable. I had to install IronTcl, set up the PATH environmental variable and the tclsh.exe by copying the tclsh86.exe from the install so that command tclsh will work.

  2. Even so, it would not compile because it assumes the tcl includes will be in the .\compat\tcl. Thus I had to copy the \include, \lib and \bin from IronTcl root foolder and put it in .\compat\tcl of the source tree.

  3. Likewise, I had to set up the .\compat\icu folder, unzipping all the files from the ICU. Otherwise, I get a linker error about unresolved external references. I think it is possible to not have to do that by adding the ICU (and Tcl) to the PATH environment variable but at that point I was just wanting to get it to work.

  4. The instruction also is not very clear on how to use the options. From the page, I thought I would be doing something like:

nmake /f Makefile.msc -DSQLITE_ENABLE_ICU

However that will not work correctly. Reviewing the MakeFile.msc and other people's notes on web, this seems to be necessary:

nmake /f ..\Makefile.msc TOP=..\ USE_ICU=1

The USE_ICU is necessary because it not only generates the -DSQLITE_ENABLE_ICU but also the necessary /I and /LIBPATH: that will correctly include the ICU files.

But this still will fail to compile the test suites because those are missing the necessary includes for some of the test codes:

line 1786-1790:

sqldiff.exe:	$(TOP)\tool\sqldiff.c $(SQLITE3C) $(SQLITE3H)	
	$(LTLINK) $(NO_WARN) $(TOP)\tool\sqldiff.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)

dbhash.exe:	$(TOP)\tool\dbhash.c $(SQLITE3C) $(SQLITE3H)	
	$(LTLINK) $(NO_WARN) $(TOP)\tool\dbhash.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)

line 1807-1808:

fuzzcheck.exe:	$(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H)	
	$(LTLINK) $(NO_WARN) $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)

To resolve those errors, I added $(LTLIBPATHS) $(LTLIBS) to end of each line which then will successfully compile the test suite with ICU enabled.

It might be possible that my modifications are not correct but I wanted to share because the instructions, especially for enabling optional features, were not very clear. Even after the above, I still get a single failing test on symlink tests so naturally I wonder if I'm still doing something wrong.

00:36 Post: Compiling sqlite3 on Windows with ICU enabled (artifact: 31b35deda9 user: Banana)

I've found that the instructions seems to be missing some crucial pieces of data and there may be some errors in the MakeFile.msc when one wants to compile the sqlite3 with ICU options enabled. There are also difference references on using ICU.

  1. The instructions fails to mention to set up the Tcl. I had made the mistake of skipping over the section #3 and going directly to section #5, thinking previous sections were POSIX-specific and not applicable. I had to install IronTcl, set up the PATH environmental variable and the tclsh.exe by copying the tclsh86.exe from the install so that command tclsh will work.

  2. Even so, it would not compile because it assumes the tcl includes will be in the .\compat\tcl. Thus I had to copy the \include, \lib and \bin from IronTcl root foolder and put it in .\compat\tcl of the source tree.

  3. Likewise, I had to set up the .\compat\icu folder, unzipping all the files from the ICU. Otherwise, I get a linker error about unresolved external references. I think it is possible to not have to do that by adding the ICU (and Tcl) to the PATH environment variable but at that point I was just wanting to get it to work.

  4. The instruction also is not very clear on how to use the options. From the page, I thought I would be doing something like:

nmake /f Makefile.msc sqlite3.c -DSQLITE_ENABLE_ICU

However that will not work correctly. Reviewing the MakeFile.msc and other people's notes on web, this seems to be necessary:

nmake /f ..\Makefile.msc test TOP=..\ USE_ICU=1

The USE_ICU is necessary because it not only generates the -DSQLITE_ENABLE_ICU but also the necessary /I and /LIBPATH: that will correctly include the ICU files.

But this still will fail to compile the test suites because those are missing the necessary includes for some of the test codes:

line 1786-1790:

sqldiff.exe:	$(TOP)\tool\sqldiff.c $(SQLITE3C) $(SQLITE3H)	
	$(LTLINK) $(NO_WARN) $(TOP)\tool\sqldiff.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)

dbhash.exe:	$(TOP)\tool\dbhash.c $(SQLITE3C) $(SQLITE3H)	
	$(LTLINK) $(NO_WARN) $(TOP)\tool\dbhash.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)

line 1807-1808:

fuzzcheck.exe:	$(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H)	
	$(LTLINK) $(NO_WARN) $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS)

To resolve those errors, I added $(LTLIBPATHS) $(LTLIBS) to end of each line which then will successfully compile the test suite with ICU enabled.

It might be possible that my modifications are not correct but I wanted to share because the instructions, especially for enabling optional features, were not very clear. Even after the above, I still get a single failing test on symlink tests so naturally I wonder if I'm still doing something wrong.