Hey there :] Happy SQLite user here,

I run sqlite in my C progrm by including the amalgamation file. Works great. However, when recompiling my program from scratch eg. after make clean, SQLite takes the longest time. Even with many files being compiled in parallel, SQLite ends up taking up extra time on one core. Thus I would like to include SQLite without the amalgamation file, all 100 something C files individually, but did not find how to do so in the documentation. Of course you could precompile a library, but I wish to remain plattform indepedant just like with the amalgamation file, but un-amalgamated.

I thought maybe I just steal all the files supplied in the "tsrc" folder during the build process to the tcl script producing the sqlite.c amalgamation. I copied those headers and c files. Here is what happens when I compile with my makefile, which simply compiles each .c file in the src folder:

[... trunkated to avoid clutter ...]
gcc -c src/fts2_porter.c -o obj/fts2_porter.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts2_tokenizer.c -o obj/fts2_tokenizer.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts2_tokenizer1.c -o obj/fts2_tokenizer1.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3.c -o obj/fts3.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_aux.c -o obj/fts3_aux.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_expr.c -o obj/fts3_expr.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_hash.c -o obj/fts3_hash.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_icu.c -o obj/fts3_icu.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_porter.c -o obj/fts3_porter.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_snippet.c -o obj/fts3_snippet.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_tokenize_vtab.c -o obj/fts3_tokenize_vtab.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_tokenizer.c -o obj/fts3_tokenizer.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_tokenizer1.c -o obj/fts3_tokenizer1.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_unicode.c -o obj/fts3_unicode.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_unicode2.c -o obj/fts3_unicode2.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts3_write.c -o obj/fts3_write.o -I inc -DSQLITE_CORE=1 
gcc -c src/fts5.c -o obj/fts5.o -I inc -DSQLITE_CORE=1 
gcc -c src/func.c -o obj/func.o -I inc -DSQLITE_CORE=1 
gcc -c src/geopoly.c -o obj/geopoly.o -I inc -DSQLITE_CORE=1 
src/geopoly.c: In function 'geopolyParseJson':
src/geopoly.c:223:12: error: 'SQLITE_OK' undeclared (first use in this function); did you mean 'SQLITE_CORE'?
  223 |   int rc = SQLITE_OK;
      |            ^~~~~~~~~
      |            SQLITE_CORE
src/geopoly.c:223:12: note: each undeclared identifier is reported only once for each function it appears in
src/geopoly.c:224:3: warning: implicit declaration of function 'memset' [-Wimplicit-function-declaration]
  224 |   memset(&s, 0, sizeof(s));
      |   ^~~~~~
src/geopoly.c:39:1: note: include '<string.h>' or provide a declaration of 'memset'
[... trunkated to avoid clutter ...]

src/geopoly.c:1765:33: error: unknown type name 'sqlite3'
 1765 | static int sqlite3_geopoly_init(sqlite3 *db){
      |                                 ^~~~~~~
make: *** [Makefile:10: obj/geopoly.o] Error 1

After compiling 45 SQLite files "src/geopoly.c:223:12: error: 'SQLITE_OK' undeclared".

Clearly the TCL script does something more than simple copy paste when producing the amalgamation file and I'm missing something. Is there something else that needs to be -D defined? What can I do to compile SQLite in such a way?

Start at the docs page, easily found at the SQLite home page. Then click on the Keyword Index in which you can find the word "amalgamation". Click on that to see advice on using or avoiding use of the amalgamation and links to build tips.

I have to say that your build method is not supported. Several methods are well supported. For those, the docs will be helpful. So will be the various makefiles.

Your quest reminds me of a day I climbed up the steep side of Diamond Head near Waikīkī, spending an arduous hour or more getting to the top where I discovered lots of tourists who had driven vehicles there. My SQLite builds take 3-25 seconds on a modern desktop, depending mainly on optimization settings. That's for the amalgamation.

Start at the docs page, easily found at the SQLite home page. Then click on the Keyword Index in which you can find the word "amalgamation".

Already read through that, indeed no mention of such a usage type.

I have to say that your build method is not supported.

Good to know, thanks! I'll read through the TCL scripts mksqlite3c and mksqlite3h just in case it ends up being some simple thing I'm missing. If it ends up working I'll respond here.

spending an arduous hour or more getting to the top where I discovered lots of tourists who had driven vehicles there

I see your point. Definetly lots of work for just a couple seconds saved via better multi core utilization... but the others on the mountain got a boring vehicle ride, whilst I'm in it for the adventure :D

However, when recompiling my program from scratch eg. after make clean, SQLite takes the longest time.

i strongly recommend making use of ccache. It reduces rebuilds of unchanged files to mere milliseconds.

For example, a full, clean rebuild of the fossil source tree takes approx. 90 seconds on my machine using a single core, and roughly half of that is compiling sqlite3.c. However, when using using ccache even clean rebuilds are lightning fast for files which have not been modified:

$ make clean
$ time make all
...
real	0m0.469s
user	0m0.366s
sys	0m0.084s

because ccache has cached the compilation results and reuses them automatically.

Making use of ccache is trivial for most projects: simply set CC=ccache cc in your login scripts, or do the following with most configure-script-driven builds:

$ ./configure CC="ccache cc"

What about the “The Split Amalgamation“ mentioned in https://www.sqlite.org/amalgamation.html?

The split amalgamation works around the limitation of 64K lines per source code file in Visual Studio. Each element of the split amalgamation must still be compiled sequentially. I think the OP's desire is to speed up compilation time (at the expense of run-time) using parallization, not to gain access to debugging information in Visual Studio, so I don't think the split amalgamation is applicable.

Eyyyyy it all works! Basically works by default. Here is what I did:

• Copy all files from tsrc
• Exclude geopoly.c from being built because This file is #include-ed onto the end of "rtree.c" so that it has access to all of the R-Tree internals. and that was the cause of the compile errors from OP
• Exclude tclsqlite.c from being built, because it apparently requires some extra sauce to properly be linked
• Define -DSQLITE_CORE=1 in the compiler and just -DSQLITE_CORE=1, -DSQLITE_AMALGAMATION=1 and -DSQLITE_PRIVATE=static cause errors
• Now everything link and runs correctly

Of course, kick out shell.c if you don't need the console interface and be careful not to overwrite your own main.c when copying the files over. I'm sure copy pasting all of tsrc includes unneeded stuff, which I'll figure out as I go along. But happy that this basically works without extra sauce. edit: corrected typo, and mixing up shell.c and main.c

@gunter_hick: glad to see I'm not the only crazy preson :]