/ Check-in [6b740c7c]
Login
SQLite training in Houston TX on 2019-11-05 (details)
Part of the 2019 Tcl Conference

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add comment to fts3rnd.test to explain how the test works.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 6b740c7cd57d618623ed028be4213dfef860054a
User & Date: dan 2009-12-11 07:07:36
Context
2009-12-11
12:29
Rationalize some code in fts3 used by optimize operations, queries of the pending-terms hash table and segment merges. Add the "INSERT INTO tbl(tbl) VALUES('optimize')" syntax. check-in: 29476da3 user: dan tags: trunk
07:07
Add comment to fts3rnd.test to explain how the test works. check-in: 6b740c7c user: dan tags: trunk
03:44
Extensive edits to the comments in the sqlite.h.in source file to identify testable statements of truth about the C-language interface. check-in: ea884e1e user: drh tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to test/fts3rnd.test.

     4      4   #    May you find forgiveness for yourself and forgive others.
     5      5   #    May you share freely, never taking more than you give.
     6      6   #
     7      7   #***********************************************************************
     8      8   #
     9      9   # Brute force (random data) tests for FTS3.
    10     10   #
           11  +
           12  +#-------------------------------------------------------------------------
           13  +#
           14  +# The FTS3 tests implemented in this file focus on testing that FTS3
           15  +# returns the correct set of documents for various types of full-text
           16  +# query. This is done using pseudo-randomly generated data and queries.
           17  +# The expected result of each query is calculated using Tcl code.
           18  +#
           19  +#   1. The database is initialized to contain a single table with three
           20  +#      columns. 100 rows are inserted into the table. Each of the three
           21  +#      values in each row is a document consisting of between 0 and 100
           22  +#      terms. Terms are selected from a vocabulary of $G(nVocab) terms.
           23  +#
           24  +#   2. The following is performed 100 times:
           25  +#
           26  +#      a. A row is inserted into the database. The row contents are 
           27  +#         generated as in step 1. The docid is a pseudo-randomly selected
           28  +#         value between 0 and 1000000.
           29  +# 
           30  +#      b. A psuedo-randomly selected row is updated. One of its columns is
           31  +#         set to contain a new document generated in the same way as the
           32  +#         documents in step 1.
           33  +# 
           34  +#      c. A psuedo-randomly selected row is deleted.
           35  +# 
           36  +#      d. For each of several types of fts3 queries, 10 SELECT queries
           37  +#         of the form:
           38  +# 
           39  +#           SELECT docid FROM <tbl> WHERE <tbl> MATCH '<query>'
           40  +# 
           41  +#         are evaluated. The results are compared to those calculated by
           42  +#         Tcl code in this file. The patterns used for the different query
           43  +#         types are:
           44  +# 
           45  +#           1.  query = <term>
           46  +#           2.  query = <prefix>
           47  +#           3.  query = "<term> <term>"
           48  +#           4.  query = "<term> <term> <term>"
           49  +#           5.  query = "<prefix> <prefix> <prefix>"
           50  +#           6.  query = <term> NEAR <term>
           51  +#           7.  query = <term> NEAR/11 <term> NEAR/11 <term>
           52  +#           8.  query = <term> OR <term>
           53  +#           9.  query = <term> NOT <term>
           54  +#           10. query = <term> AND <term>
           55  +#           11. query = <term> NEAR <term> OR <term> NEAR <term>
           56  +#           12. query = <term> NEAR <term> NOT <term> NEAR <term>
           57  +#           13. query = <term> NEAR <term> AND <term> NEAR <term>
           58  +# 
           59  +#         where <term> is a term psuedo-randomly selected from the vocabulary
           60  +#         and prefix is the first 2 characters of such a term followed by
           61  +#         a "*" character.
           62  +#     
           63  +#      Every second iteration, steps (a) through (d) above are performed
           64  +#      within a single transaction. This forces the queries in (d) to
           65  +#      read data from both the database and the in-memory hash table
           66  +#      that caches the full-text index entries created by steps (a), (b)
           67  +#      and (c) until the transaction is committed.
           68  +#
           69  +# The procedure above is run 5 times, using advisory fts3 node sizes of 50,
           70  +# 500, 1000 and 2000 bytes.
           71  +#
           72  +# After the test using an advisory node-size of 50, an OOM test is run using
           73  +# the database. This test is similar to step (d) above, except that it tests
           74  +# the effects of transient and persistent OOM conditions encountered while
           75  +# executing each query.
           76  +#
    11     77   
    12     78   set testdir [file dirname $argv0]
    13     79   source $testdir/tester.tcl
    14     80   
    15     81   # If this build does not include FTS3, skip the tests in this file.
    16     82   #
    17     83   ifcapable !fts3 { finish_test ; return }
    18     84   source $testdir/fts3_common.tcl
    19     85   
           86  +set G(nVocab) 100
           87  +
    20     88   set nVocab 100
    21     89   set lVocab [list]
    22     90   
    23     91   expr srand(0)
    24         -
    25     92   
    26     93   # Generate a vocabulary of nVocab words. Each word is 3 characters long.
    27     94   #
    28     95   set lChar {a b c d e f g h i j k l m n o p q r s t u v w x y z}
    29     96   for {set i 0} {$i < $nVocab} {incr i} {
    30     97     set    word [lindex $lChar [expr int(rand()*26)]]
    31     98     append word [lindex $lChar [expr int(rand()*26)]]