/ Check-in [18d4e258]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Change README to README.md and expand it. Remove unrelated and obsolete files.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 18d4e258c45c32984b23d97b896a761eeef2dbdf
User & Date: drh 2014-02-13 19:10:24
Context
2014-02-13
19:27
Ensure that if the "psow=0" URI option or FCNTL_POWERSAFE_OVERWRITE file-control is used to clear the power-safe overwrite flag, extra padding frames are added to the WAL file. check-in: 48c821fd user: dan tags: trunk
19:10
Change README to README.md and expand it. Remove unrelated and obsolete files. check-in: 18d4e258 user: drh tags: trunk
18:46
Avoid passing a flags argument with the internal WAL_SYNC_TRANSACTIONS bit set when calling sqlite3OsSync(). check-in: e3b79e92 user: dan tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Name change from README to README.md.

     1         -This directory contains source code to 
            1  +<h1 align="center">SQLite Source Repository</h1>
     2      2   
     3         -    SQLite: An Embeddable SQL Database Engine
            3  +This repository contains the complete source code for the SQLite database
            4  +engine.  Some test scripts are also include.  However, many other test scripts
            5  +and most of the documentation are managed separately.
     4      6   
     5         -To compile the project, first create a directory in which to place
            7  +## Compiling
            8  +
            9  +First create a directory in which to place
     6     10   the build products.  It is recommended, but not required, that the
     7     11   build directory be separate from the source directory.  Cd into the
     8     12   build directory and then from the build directory run the configure
     9     13   script found at the root of the source tree.  Then run "make".
    10     14   
    11     15   For example:
    12     16   
    13     17       tar xzf sqlite.tar.gz    ;#  Unpack the source tree into "sqlite"
    14     18       mkdir bld                ;#  Build will occur in a sibling directory
    15     19       cd bld                   ;#  Change to the build directory
    16     20       ../sqlite/configure      ;#  Run the configure script
    17     21       make                     ;#  Run the makefile.
    18         -    make install             ;#  (Optional) Install the build products
           22  +    make sqlite3.c           ;#  Build the "amalgamation" source file
           23  +    make test                ;#  Run some tests (requires TCL)
           24  +
           25  +See the makefile for additional targets.
    19     26   
    20     27   The configure script uses autoconf 2.61 and libtool.  If the configure
    21     28   script does not work out for you, there is a generic makefile named
    22     29   "Makefile.linux-gcc" in the top directory of the source tree that you
    23     30   can copy and edit to suit your needs.  Comments on the generic makefile
    24     31   show what changes are needed.
    25     32   
    26         -The linux binaries on the website are created using the generic makefile,
    27         -not the configure script.  The windows binaries on the website are created
    28         -using MinGW32 configured as a cross-compiler running under Linux.  For 
    29         -details, see the ./publish.sh script at the top-level of the source tree.
    30         -The developers do not use teh configure script.
    31         -
    32     33   SQLite does not require TCL to run, but a TCL installation is required
    33     34   by the makefiles.  SQLite contains a lot of generated code and TCL is
    34     35   used to do much of that code generation.  The makefile also requires
    35     36   AWK.
    36     37   
    37         -Contacts:
           38  +## Source Code Tour
           39  +
           40  +Most of the core source files are in the **src/** subdirectory.  But
           41  +src/ also contains files used to build the "testfixture" test harness;
           42  +those file all begin with "test".  And src/ contains the "shell.c" file
           43  +which is the main program for the "sqlite3.exe" command-line shell and
           44  +the "tclsqlite.c" file which implements the bindings to SQLite from the
           45  +TCL programming language.  (Historical note:  SQLite began as a TCL
           46  +extension and only later escaped to the wild as an independent library.)
           47  +
           48  +Test scripts and programs are found in the **test/** subdirectory.
           49  +There are other test suites for SQLite (see 
           50  +[How SQLite Is Tested](http://www.sqlite.org/testing.html))
           51  +but those other test suites are
           52  +in separate source repositories.
           53  +
           54  +The **ext/** subdirectory contains code for extensions.  The
           55  +Full-text search engine is in **ext/fts3**.  The R-Tree engine is in
           56  +**ext/rtree**.  The **ext/misc** subdirectory contains a number of
           57  +smaller, single-file extensions, such as a REGEXP operator.
           58  +
           59  +The **tool/** subdirectory contains various scripts and programs used
           60  +for building generated source code files or for testing or for generating
           61  +accessory programs such as "sqlite3_analyzer(.exe)".
           62  +
           63  +### Generated Source Code Files
           64  +
           65  +Several of the C-language source files used by SQLite are generated from
           66  +other sources rather than being typed in manually by a programmer.  This
           67  +section will summarize those automatically-generated files.  To create all
           68  +of the automatically-generated files, simply run "make target&#95;source".
           69  +The "target&#95;source" make target will create a subdirectory "tsrc/" and
           70  +fill it with all the source files needed to build SQLite, both 
           71  +manually-edited files and automatically-generated files.
           72  +
           73  +The SQLite interface is defined by the **sqlite3.h** header file, which is
           74  +generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION.  The
           75  +TCL script at tool/mksqlite3h.tcl does the conversion.  The manifest.uuid
           76  +file contains the SHA1 hash of the particular check-in and is used to generate
           77  +the SQLITE_SOURCE_ID macro.  The VERSION file contains the current SQLite
           78  +version number.  The sqlite3.h header is really just a copy of src/sqlite.h.in
           79  +with the source-id and version number inserted at just the right spots.
           80  +Note that comment text in the sqlite3.h file is used to generate much of
           81  +the SQLite API documentation.  The TCL scripts used to generate that
           82  +documentation are in a separate source repository.
           83  +
           84  +The SQL language parser is **parse.c** which is generate from a grammar in
           85  +the src/parse.y file.  The conversion of "parse.y" into "parse.c" is done
           86  +by the [lemon](./doc/lemon.html) LALR(1) parser generator.  The source code 
           87  +for lemon is at tool/lemon.c.  Lemon uses a
           88  +template for generating its parser.  A generic template is in tool/lempar.c,
           89  +but SQLite uses a slightly modified template found in src/lempar.c.
           90  +
           91  +Lemon also generates the **parse.h** header file, at the same time it
           92  +generates parse.c. But the parse.h header file is
           93  +modified further (to add additional symbols) using the ./addopcodes.awk
           94  +AWK script.
           95  +
           96  +The **opcodes.h** header file contains macros that define the numbers
           97  +corresponding to opcodes in the "VDBE" virtual machine.  The opcodes.h
           98  +file is generated by the scanning the src/vdbe.c source file.  The
           99  +AWK script at ./mkopcodeh.awk does this scan and generates opcodes.h.
          100  +A second AWK script, ./mkopcodec.awk, then scans opcodes.h to generate
          101  +the **opcodes.c** source file, which contains a reverse mapping from
          102  +opcode-number to opcode-name that is used for EXPLAIN output.
          103  +
          104  +The **keywordhash.h** header file contains the definition of a hash table
          105  +that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into
          106  +the numeric codes used by the parse.c parser.  The keywordhash.h file is
          107  +generated by a C-language program at tool mkkeywordhash.c.
          108  +
          109  +### The Amalgamation
          110  +
          111  +All of the individual C source code and header files (both manually-edited
          112  +and automatically-generated) can be combined into a single big source file
          113  +**sqlite3.c** called "the amalgamation".  The amalgamation is the recommended
          114  +way of using SQLite in a larger application.  Combining all individual
          115  +source code files into a single big source code file allows the C compiler
          116  +to perform more cross-procedure analysis and generate better code.  SQLite
          117  +runs about 5% faster when compiled from the amalgamation versus when compiled
          118  +from individual source files.
          119  +
          120  +The amalgamation is generated from the tool/mksqlite3c.tcl TCL script.
          121  +First, all of the individual source files must be gathered into the tsrc/
          122  +subdirectory (using the equivalent of "make target_source") then the
          123  +tool/mksqlite3c.tcl script is run to copy them all together in just the
          124  +right order while resolving internal "#include" references.
          125  +
          126  +The amalgamation source file is more than 100K lines long.  Some symbolic
          127  +debuggers (most notably MSVC) are unable to deal with files longer than 64K
          128  +lines.  To work around this, a separate TCL script, tool/split-sqlite3c.tcl,
          129  +can be run on the amalgamation to break it up into a single small C file
          130  +called **sqlite3-all.c** that does #include on about five other files
          131  +named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-5.c**.  In this way,
          132  +all of the source code is contained within a single translation unit so
          133  +that the compiler can do extra cross-procedure optimization, but no 
          134  +individual source file exceeds 32K lines in length.
          135  +
          136  +## How It All Fits Together
          137  +
          138  +SQLite is modular in design.
          139  +See the [architectural description](http://www.sqlite.org/arch.html) 
          140  +for details. Other documents that are useful in
          141  +(helping to understand how SQLite works include the
          142  +[file format](http://www.sqlite.org/fileformat2.html) description,
          143  +the [virtual machine](http://www.sqlite.org/vdbe.html) that runs
          144  +prepared statements, the description of 
          145  +[how transactions work](http://www.sqlite.org/atomiccommit.html), and
          146  +the [overview of the query planner](http://www.sqlite.org/optoverview.html).
          147  +
          148  +Unfortunately, years of effort have gone into optimizating SQLite, both
          149  +for small size and high performance.  And optimizations tend to result in
          150  +complex code.  So there is a lot of complexity in the SQLite implementation.
          151  +
          152  +Key files:
          153  +
          154  +  *  **sqlite3.h** - This file defines the public interface to the SQLite
          155  +     library.  Readers will need to be familiar with this interface before
          156  +     trying to understand how the library works internally.
          157  +
          158  +  *  **sqliteInt.h** - this header file defines many of the data objects
          159  +     used internally by SQLite.
          160  +
          161  +  *  **parse.y** - This file describes the LALR(1) grammer that SQLite uses
          162  +     to parse SQL statements, and the actions that are taken at each stop
          163  +     in the parsing process.
          164  +
          165  +  *  **vdbe.c** - This file implements the virtual machine that runs
          166  +     prepared statements.  There are various helper files whose names
          167  +     begin with "vdbe".  The VDBE has access to the vdbeInt.h header file
          168  +     which defines internal data objects.  The rest of SQLite interacts
          169  +     with the VDBE through an interface defined by vdbe.h.
          170  +
          171  +  *  **where.c** - This file analyzes the WHERE clause and generates
          172  +     virtual machine code to run queries efficiently.  This file is
          173  +     sometimes called the "query optimizer".  It has its own private
          174  +     header file, whereInt.h, that defines data objects used internally.
          175  +
          176  +  *  **btree.c** - This file contains the implementation of the B-Tree
          177  +     storage engine used by SQLite.
          178  +
          179  +  *  **pager.c** - This file contains the "pager" implementation, the
          180  +     module that implements transactions.
          181  +
          182  +  *  **os_unix.c** and **os_win.c** - These two files implement the interface
          183  +     between SQLite and the underlying operating system using the run-time
          184  +     pluggable VFS interface.
          185  +  
          186  +
          187  +## Contacts
    38    188   
    39         -   http://www.sqlite.org/
          189  +The main SQLite webpage is [http://www.sqlite.org/](http://www.sqlite.org/)
          190  +with geographically distributed backup servers at
          191  +[http://www2.sqlite.org/](http://www2.sqlite.org) and
          192  +[http://www3.sqlite.org/](http://www3.sqlite.org).

Deleted mkdll.sh.

     1         -#!/bin/sh
     2         -#
     3         -# This script is used to compile SQLite into a DLL.
     4         -#
     5         -# Two separate DLLs are generated.  "sqlite3.dll" is the core
     6         -# library.  "tclsqlite3.dll" contains the TCL bindings and is the
     7         -# library that is loaded into TCL in order to run SQLite.
     8         -#
     9         -make sqlite3.c
    10         -PATH=$PATH:/opt/mingw/bin
    11         -TCLDIR=/home/drh/tcltk/846/win/846win
    12         -TCLSTUBLIB=$TCLDIR/libtcl84stub.a
    13         -OPTS='-DUSE_TCL_STUBS=1 -DBUILD_sqlite=1 -DSQLITE_OS_WIN=1'
    14         -OPTS="$OPTS -DSQLITE_THREADSAFE=1"
    15         -OPTS="$OPTS -DSQLITE_ENABLE_FTS3=1"
    16         -OPTS="$OPTS -DSQLITE_ENABLE_RTREE=1"
    17         -OPTS="$OPTS -DSQLITE_ENABLE_COLUMN_METADATA=1"
    18         -CC="i386-mingw32msvc-gcc -Os $OPTS -Itsrc -I$TCLDIR"
    19         -NM="i386-mingw32msvc-nm"
    20         -CMD="$CC -c sqlite3.c"
    21         -echo $CMD
    22         -$CMD
    23         -CMD="$CC -c tclsqlite3.c"
    24         -echo $CMD
    25         -$CMD
    26         -echo 'EXPORTS' >tclsqlite3.def
    27         -$NM tclsqlite3.o | grep ' T ' >temp1
    28         -grep '_Init$' temp1 >temp2
    29         -grep '_SafeInit$' temp1 >>temp2
    30         -grep ' T _sqlite3_' temp1 >>temp2
    31         -echo 'EXPORTS' >tclsqlite3.def
    32         -sed 's/^.* T _//' temp2 | sort | uniq >>tclsqlite3.def
    33         -i386-mingw32msvc-dllwrap \
    34         -     --def tclsqlite3.def -v --export-all \
    35         -     --driver-name i386-mingw32msvc-gcc \
    36         -     --dlltool-name i386-mingw32msvc-dlltool \
    37         -     --as i386-mingw32msvc-as \
    38         -     --target i386-mingw32 \
    39         -     -dllname tclsqlite3.dll -lmsvcrt tclsqlite3.o $TCLSTUBLIB
    40         -$NM sqlite3.o | grep ' T ' >temp1
    41         -echo 'EXPORTS' >sqlite3.def
    42         -grep ' _sqlite3_' temp1 | sed 's/^.* _//' >>sqlite3.def
    43         -i386-mingw32msvc-dllwrap \
    44         -     --def sqlite3.def -v --export-all \
    45         -     --driver-name i386-mingw32msvc-gcc \
    46         -     --dlltool-name i386-mingw32msvc-dlltool \
    47         -     --as i386-mingw32msvc-as \
    48         -     --target i386-mingw32 \
    49         -     -dllname sqlite3.dll -lmsvcrt sqlite3.o

Deleted mkextu.sh.

     1         -#!/bin/sh
     2         -#
     3         -# This script is used to compile SQLite into a shared library on Linux.
     4         -#
     5         -# Two separate shared libraries are generated.  "sqlite3.so" is the core
     6         -# library.  "tclsqlite3.so" contains the TCL bindings and is the
     7         -# library that is loaded into TCL in order to run SQLite.
     8         -#
     9         -CFLAGS=-O2 -Wall
    10         -make fts2amal.c
    11         -echo gcc $CFLAGS -shared fts2amal.c -o fts2.so
    12         -gcc $CFLAGS -shared fts2amal.c -o fts2.so
    13         -strip fts2.so

Deleted mkextw.sh.

     1         -#!/bin/sh
     2         -#
     3         -# This script is used to compile SQLite extensions into DLLs.
     4         -#
     5         -make fts2amal.c
     6         -PATH=$PATH:/opt/mingw/bin
     7         -OPTS='-DTHREADSAFE=1 -DBUILD_sqlite=1 -DSQLITE_OS_WIN=1'
     8         -CC="i386-mingw32msvc-gcc -O2 $OPTS -Itsrc"
     9         -NM="i386-mingw32msvc-nm"
    10         -CMD="$CC -c fts2amal.c"
    11         -echo $CMD
    12         -$CMD
    13         -echo 'EXPORTS' >fts2.def
    14         -echo 'sqlite3_fts2_init' >>fts2.def
    15         -i386-mingw32msvc-dllwrap \
    16         -     --def fts2.def -v --export-all \
    17         -     --driver-name i386-mingw32msvc-gcc \
    18         -     --dlltool-name i386-mingw32msvc-dlltool \
    19         -     --as i386-mingw32msvc-as \
    20         -     --target i386-mingw32 \
    21         -     -dllname fts2.dll -lmsvcrt fts2amal.o
    22         -zip fts2dll.zip fts2.dll fts2.def