Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes to ext/fts3/README.tokenizers.

    48     48     returned. If only one argument is passed, a pointer to the tokenizer
    49     49     implementation currently registered as <tokenizer-name> is returned,
    50     50     encoded as a blob. Or, if no such tokenizer exists, an SQL exception
    51     51     (error) is raised.
    52     52   
    53     53     SECURITY: If the fts3 extension is used in an environment where potentially
    54     54       malicious users may execute arbitrary SQL (i.e. gears), they should be
    55         -    prevented from invoking the fts3_tokenizer() function, possibly using the
    56         -    authorisation callback.
           55  +    prevented from invoking the fts3_tokenizer() function.  The
           56  +    fts3_tokenizer() function is disabled by default. It is only enabled
           57  +    by SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER. Do not enable it in
           58  +    security sensitive environments.
    57     59   
    58     60     See "Sample code" below for an example of calling the fts3_tokenizer()
    59     61     function from C code.
    60     62   
    61     63   3. ICU Library Tokenizers
    62     64   
    63     65     If this extension is compiled with the SQLITE_ENABLE_ICU pre-processor 

Changes to ext/fts3/fts3_tokenizer.c.

   102    102       if( !pPtr ){
   103    103         char *zErr = sqlite3_mprintf("unknown tokenizer: %s", zName);
   104    104         sqlite3_result_error(context, zErr, -1);
   105    105         sqlite3_free(zErr);
   106    106         return;
   107    107       }
   108    108     }
   109         -  sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT);
          109  +  if( fts3TokenizerEnabled(context) ){
          110  +    sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT);
          111  +  }
   110    112   }
   111    113   
   112    114   int sqlite3Fts3IsIdChar(char c){
   113    115     static const char isFtsIdChar[] = {
   114    116         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,  /* 0x */
   115    117         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,  /* 1x */
   116    118         0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,  /* 2x */

Changes to src/sqlite.h.in.

  2082   2082   ** The second parameter is a pointer to an integer into which
  2083   2083   ** is written 0 or 1 to indicate whether triggers are disabled or enabled
  2084   2084   ** following this call.  The second parameter may be a NULL pointer, in
  2085   2085   ** which case the trigger setting is not reported back. </dd>
  2086   2086   **
  2087   2087   ** [[SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER]]
  2088   2088   ** <dt>SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER</dt>
  2089         -** <dd> ^This option is used to enable or disable the two-argument
  2090         -** version of the [fts3_tokenizer()] function which is part of the
         2089  +** <dd> ^This option is used to enable or disable the
         2090  +** [fts3_tokenizer()] function which is part of the
  2091   2091   ** [FTS3] full-text search engine extension.
  2092   2092   ** There should be two additional arguments.
  2093   2093   ** The first argument is an integer which is 0 to disable fts3_tokenizer() or
  2094   2094   ** positive to enable fts3_tokenizer() or negative to leave the setting
  2095   2095   ** unchanged.
  2096   2096   ** The second parameter is a pointer to an integer into which
  2097   2097   ** is written 0 or 1 to indicate whether fts3_tokenizer is disabled or enabled

Changes to test/fts3atoken.test.

   103    103   } {1 {unknown tokenizer: nosuchtokenizer}}
   104    104   
   105    105   #--------------------------------------------------------------------------
   106    106   # Test cases fts3atoken-3.* test the three built-in tokenizers with a
   107    107   # simple input string via the built-in test function. This is as much
   108    108   # to test the test function as the tokenizer implementations.
   109    109   #
          110  +sqlite3_db_config db SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER 1
   110    111   do_test fts3atoken-3.1 {
   111    112     execsql {
   112    113       SELECT fts3_tokenizer_test('simple', 'I don''t see how');
   113    114     }
   114    115   } {{0 i I 1 don don 2 t t 3 see see 4 how how}}
   115    116   do_test fts3atoken-3.2 {
   116    117     execsql {