SQLite Forum

Anyone know how to compile RTree and Geopoly as shared libraries on macOS?
Login

Anyone know how to compile RTree and Geopoly as shared libraries on macOS?

(1) By Simon Willison (simonw) on 2020-08-10 03:01:59 [link] [source]

I'd love to be able to easily use the RTree and Geopoly modules on systems that already have SQLite installed but didn't include those in their installation.

Ideally I want to build a .so module that I can load like this (in Python):

import sqlite3
conn = sqlite3.connect("places.db")
conn.enable_load_extension(True)
conn.load_extension("rtree.so")
conn.load_extension("geopoly.so")

Does anyone know the right incantations on macOS (and ideally on Linux too) to compile the source code for these modules into a library that I can load in this way?

(2) By anonymous on 2020-08-11 08:25:56 in reply to 1 [link] [source]

Does this help? It can be easily found by opening SQLite documentation and searching for "extension".

(3) By Simon Willison (simonw) on 2020-08-11 14:52:41 in reply to 2 [link] [source]

Unfortunately it doesn't. I read that and tried to figure out how to apply that to compiling the rtree and geopoly modules for macOS and could not figure out how to do it - hence my question here! I'm hoping someone who's better at working with C and macOS can help me work out exactly what I need to do.

(4) By Larry Brasfield (LarryBrasfield) on 2020-08-11 15:23:28 in reply to 3 [source]

Look at Run-Time Loadable Extensions, section 3. I found this by doing the suggested search on "extension" and following the most obviously applicable result.

(5) By anonymous on 2020-08-11 15:29:10 in reply to 3 [link] [source]

It would help a lot if you told us what you have already tried. (Please always do so when asking for technical help.)

For example, if I download rtree.c and sqlite3rtree.h and run gcc -g -fPIC -shared rtree.c -o rtree.so, I get an rtree.so that I can't load because it relies on sqlite3GetToken, which is not exported from my system libsqlite3.so. If I additionally download geopoly.c and run gcc -DSQLITE_ENABLE_GEOPOLY -g -fPIC -shared rtree.c -o rtree.so -lsqlite3, I get compilation errors because my system SQLite is too old. I am not sure how to solve the former problem (why isn't the symbol exported?), but the latter requires me to update SQLite (or recompile it manually, but then I'd just compile the amalgamation with -DSQLITE_ENABLE_RTREE and -DSQLITE_ENABLE_GEOPOLY).

But that's my system. Which errors do you get?

(6) By Scott Perry (numist) on 2020-08-11 16:04:22 in reply to 5 [link] [source]

For security reasons, the libsqlite3.dylib provided by macOS ships with extension loading disabled.

At a higher level, the concept of global extensions is incompatible with a world in which applications contain linkages written by different authors, each with their own opinions about how SQLite should work for them. We've even seen a couple libraries call sqlite3_config(SQLITE_CONFIG_SINGLETHREAD, 0), to the chagrin of everyone else in the process.

As for why a built in module isn't available on the system library, well… I'll get on that :)

(7) By Keith Medcalf (kmedcalf) on 2020-08-11 20:43:05 in reply to 1 [link] [source]

The RTree/Geopoly extension can only be compiled as part of the core because it requires access to an internal function within the SQLite3 core. It cannot be compiled as a "loadable extension" for this reason.

You need to recompile the entire SQLite3 core shared library with the appropriate defines to include the RTREE and Geopoly as part of the core.

(8) By Simon Willison (simonw) on 2020-08-12 05:44:25 in reply to 7 [link] [source]

Thanks! You've saved me a lot of time. I've managed to compile them using the amalgamation so I guess I'll stick with that.