In commit 95dca8d0c, fts5TriTokenize() in ext/fts5/fts5_tokenize.c was patched to prevent the trigram tokenizer from returning tokens that contain embedded nul characters. There is similar logic in fts5UnicodeTokenize(), so I think there should be a check on iCode after READ_UTF8() too.

    while( 1 ){
      if( zCsr>=zTerm ) goto tokenize_done;
      if( *zCsr & 0x80 ) {
        /* A character outside of the ascii range. Skip past it if it is
        ** a separator character. Or break out of the loop if it is not. */
        is = zCsr - (unsigned char*)pText;
        READ_UTF8(zCsr, zTerm, iCode);
        if( fts5UnicodeIsAlnum(p, iCode) ){
          goto non_ascii_tokenchar;
        }
      }else{
        if( a[*zCsr] ){
          is = zCsr - (unsigned char*)pText;
          goto ascii_tokenchar;
        }
        zCsr++;
      }
    }

To clarify: This error is not a vulnerability. No memory errors or other exploit opportunities occur due to this error. The error causes FTS5 to sometimes return an incorrect result. But the error cannot be exploited by an attacker to compromise the system, to our knowledge.

This clarification is added because a third-party has written a (bogus) CVE against this error.

Hi Richard,

I am developer, working in company that uses SQLite. Currently, we are using SQLite 3.23.1. Meld reported CVE-2021-20223 issue to us. I am trying to implement patch on this version, but it is not so straightforward because it is quit old version in regards to the version for which the patch is already implemented and lots of things had changed in between.

I saw this comment of yours and wondering, if this is not vulnerability, then patch is not necessary. Can you please confirm this? Since I am not aware of your role, I am wondering, can I rely on your comment.

Thanks in advance!

Robert

Can you please confirm this? Since I am not aware of your role, I am wondering, can I rely on your comment.

Richard Hipp is this project's main architect and lead developer. His assessment about this being a non-vulnerability is the most authoritative one you can possibly get.

Hi Stephan,

Well, that's the answer I was looking for :). Thanks a lot for quick response!

Best regards, Robert