SQLite Forum

Last chance to beta-test and comment on 3.41.0
Login

Last chance to beta-test and comment on 3.41.0

(1.1) By Richard Hipp (drh) on 2023-02-16 21:22:46 edited from 1.0 [link] [source]

The 3.41.0 release of SQLite will likely appear within a week or so. The release checklist is hot. When it goes all-green, the release will occur.

Please do beta-testing using the latest trunk check-in (not the various branches we've been working with for the past couple of days) or using the pre-release snapshot.

See https://sqlite.org/draft/ for the latest draft of the 3.41.0 website. Dates thereon are approximate.

Report any issues or concerns as follow-ups to this forum thread, or by direct email to drh at sqlite dot org.

Almost forgot: The two query optimizer enhancements that are currently pending will not land on trunk until the next release cycle at the earliest.

(2) By Mark Lawrence (mark) on 2023-02-17 09:41:22 in reply to 1.1 [link] [source]

The releaselog mentions base64() and base85() functions added to the CLI by default, but there is no documentation for those that I could find in the draft website.

(3) By ddevienne on 2023-02-17 10:00:46 in reply to 1.1 [source]

The doc from https://sqlite.org/draft/carray.html mentions iovec, but does not show its structure,
and clicking the link to the C file goes to the wrong one not from the draft site. Could the doc be more explicit?

I suppose the length of each blob must be specified, right?
Do the blob need to be in contiguous memory, or scattered about?

If the latter, I'd very much appreciate the ability to do the same for strings, not just blobs,
to be able to using carray for a std::vector<std::string_view> for example, i.e. non-null terminated strings.

(4) By Stephan Beal (stephan) on 2023-02-17 12:42:51 in reply to 3 [link] [source]

The doc from https://sqlite.org/draft/carray.html mentions iovec, but does not show its structure,

iovec is a POSIX-defined type. See:

(5) By ddevienne on 2023-02-17 12:47:52 in reply to 4 [link] [source]

But isn't SQLite's code, including extensions, cross-platform? I'm confused...

Because a quick search found an SO post that hints it isn't.

(6) By Stephan Beal (stephan) on 2023-02-17 12:56:47 in reply to 5 [link] [source]

But isn't SQLite's code, including extensions, cross-platform? I'm confused...

There's a hand-written impl in place for the Windows builds, but it's a standardized 2-member struct so doesn't require custom docs.

The POSIX i/o methods which use that struct aren't actually used by carray. The struct is only used to hold the data.

(7.1) By ddevienne on 2023-02-17 13:57:29 edited from 7.0 in reply to 6 [link] [source]

OK, thanks for the pointer.

That's unfortunately hardly convenient to use from the Windows side though,
since that definition is private to the extension code, and thus will need
to be duplicated in client-code that wants to be portable to Windows (ODR violation anyone?).

A small nuisance perhaps to some, but still.
Also uses size_t, which SQLite APIs normally don't use,
and for good reason, having a variable size depending on the build.

Would be much more friendly to define an SQLite specific little struct for scatter-gather IO,
in the same spirit as struct iovec (e.g. `sqlite3_iovec), in a (new?) header part of the almagamation's sqlite3.h,
thus easily used in code from any platform, and use it rather than a Posix'ism not supported on Windows.

Then the new struct iovec datatype of carray could be changed to blob[],
with text[] also added, to allow scatter-gather on both text and blob values,
without requiring null-terminated strings. See below for what I mean.

I can of course copy/paste the carray.c and change it myself,
but given that the changes I suggest are easily done if not trivial,
I hope they'll be considered. The use of Posix' struct iovec feels very un-SQLite to me.

        case CARRAY_TEXT: {
          const char **p = (const char**)pCur->pPtr;
          sqlite3_result_text(ctx, p[pCur->iRowid-1], -1, SQLITE_TRANSIENT);
          return SQLITE_OK;
        }
        case CARRAY_TEXT_ARRAY: {
          const struct iovec *p = (sqlite3_iovec*)pCur->pPtr;
          sqlite3_result_text(ctx, p[pCur->iRowid-1].iov_base, // With maybe a cast here
                               (int)p[pCur->iRowid-1].iov_len, SQLITE_TRANSIENT);
          return SQLITE_OK;
        }
        case CARRAY_BLOB_ARRAY: {
          const struct iovec *p = (sqlite3_iovec*)pCur->pPtr;
          sqlite3_result_blob(ctx, p[pCur->iRowid-1].iov_base,
                               (int)p[pCur->iRowid-1].iov_len, SQLITE_TRANSIENT);
          return SQLITE_OK;
        }

(8) By Stephan Beal (stephan) on 2023-02-17 14:08:33 in reply to 7.1 [link] [source]

The use of Posix' struct iovec feels very un-SQLite to me.

It was done that way per specific request from a customer.

Any interface changes are out of scope for the 3.41 release (only high-priority fixes and doc tweaks are permitted at this point) but your suggestions will be considered for the next release cycle.

(9) By ddevienne on 2023-02-17 14:17:07 in reply to 8 [link] [source]

It was done that way per specific request from a customer.

Hmmm...

your suggestions will be considered for the next release cycle

Consider me placated I guess. Thanks Stephan.