SQLite Forum

how to force python to use recent sqlite3 version
Login

how to force python to use recent sqlite3 version

(1) By Petr (PetrSwetr) on 2020-05-01 01:53:09 [link] [source]

Is there a way to force python to use the recent way of sqlite3?

$ python -c "import sqlite3; print(sqlite3.sqlite_version)" 3.22.0

$ sqlite3 -version 3.30.0 2019-10-04 15:03:17 c20a35336432025445f9f7e289d0cc3e4003fb17f45a4ce74c6269c407c6e09f

$ whereis sqlite3 sqlite3: /usr/local/bin/sqlite3

petr@husova:~$ python Python 2.7.17 (default, Apr 15 2020, 17:20:14) [GCC 7.5.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import sqlite3 >>> print(sqlite3.file) /usr/lib/python2.7/sqlite3/init.pyc

(2) By beetlejuice (coleifer) on 2020-05-01 02:21:57 in reply to 1 [link] [source]

(3.1) By Larry Brasfield (LarryBrasfield) on 2020-05-01 02:45:01 edited from 3.0 in reply to 1 [link] [source]

It is intriguing that you are using Python 2.7, which has been at end-of-life for many months and reached End of support 4 months ago, yet wish to use a modern version of SQLite with it.

That said, I doubt official Python 2.7 builds are going to be appearing at all, let alone with updated libraries. But it is open source; you can build it yourself and its libraries (which I think are all open source.) I'm sure you could find the SQLite adapter code for Python 2.7 and build just that, if you were sufficiently motivated.

Another option would be to migrate to Python 3. It's not particularly difficult, as it mainly involves leaving behind some crufty constructs that were not as orthogonal or Pythonic as would suit those who developed that creed. It might be easier than fighting the drawn-out demise of Python 2.

You might also consider using Roger Binn's ASPW library. He appears to be still making it for older Python versions, using quite modern SQLite versions.

(4) By Petr (PetrSwetr) on 2020-05-01 08:29:43 in reply to 3.1 [link] [source]

I understand your reply about Python 2.7 and Python 3

The case is, the problem is not connected with Python 2.7 only.

petr@husova:~$ python3 Python 3.6.9 (default, Apr 18 2020, 01:56:04) [GCC 8.4.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sqlite3 >>> sqlite3.sqlite_version '3.22.0' >>> print(sqlite3.file) /usr/lib/python3.6/sqlite3/init.py

(5) By Petr (PetrSwetr) on 2020-05-01 08:31:32 in reply to 4 [link] [source]

petr@husova:~$ python3
Python 3.6.9 (default, Apr 18 2020, 01:56:04) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
>>> sqlite3.sqlite_version
'3.22.0'
>>> print(sqlite3.__file__)
/usr/lib/python3.6/sqlite3/__init__.py
>>>

(6) By Keith Medcalf (kmedcalf) on 2020-05-01 09:11:02 in reply to 1 [link] [source]

Find the sqlite3.so that python is using.
Replace it with the version that you want to be used.
Sacrifice three chickens and a turkey and recite whatever magical incantation is necessary for your system to recognize the change your have made.

(7) By Petr (PetrSwetr) on 2020-05-01 13:03:23 in reply to 6 [link] [source]

Successfully solved! Thanks!

feedback for others facing the same problem:

* compile last version of sqlite3
* backup the file /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
* from the compilation directory copy the file ~/sqlite-autoconf-3310100/.libs/libsqlite3.so.0.8.6 to the /usr/lib/x86_64-linux-gnu/
* install compiled version (sudo make install )

Finally python and system have the same version of sqlite3

(8) By Warren Young (wyoung) on 2020-05-01 14:02:03 in reply to 7 [source]

That method will fail every time the OS provider delivers a new binary package, because installing the upgrade will overwrite your library.

You'd do better to either:

  1. Remove the OS package so that all library users switch over to the copy in /usr/local/lib instead. This may mean suppressing warnings from your package management system.

  2. Do what you did, but mark the package non-upgradeable in your OS's package manager configuration. It is then your responsibility to keep the package maintained.

  3. Clone your OS's package scripts and substitute the new version in. For a Red Hattish system, this means taking the *.spec file, editing it with the updated versions, and then doing an rpmbuild --rebuild on the new spec file + tarball combo. For Debian type Linuxes, it's been too long since I did it to give useful advice. However you arrange it, this method avoids the problem because your version will forever be greater than whatever your OS provider offers.

  4. Find a sufficiently closely-related OS that ships a newer compatible package, rebuild its source package for your OS, and use that instead. e.g. Ubuntu for Debian, Fedora for CentOS, etc.

Personally, I'd pick #3, but that's in part because I've maintained OS packages before, so it's not a new skill for me.

(9) By Larry Brasfield (LarryBrasfield) on 2020-05-01 14:35:28 in reply to 7 [link] [source]

First, I hope you are happy with this solution, and remain so. I say the following as a warning to people who might emulate it.

The DLL versioned-naming scheme common on Unix (and Linux) systems is a reasonable and effective way of allowing DLLs to be shared, where they can be, and still avoid the problem of incompatible DLLs being loaded (or load attempts failing) as specific DLL interfaces evolve.

The above "solution" would be consistent with that scheme and safe to employ if the following are true: (1) the substitute DLL has all of the entry points as the one replaced; and (2) those entry points work the same way as before.

Those two requirements are likely to be met by building SQLite with the same feature set selected as was selected for the replaced DLL, or a superset. This is because the SQLite interface (aka the set of entry points and their behavior) tends to only grow and remain highly backward compatible. (However, "tends" does not equate to "always".)

I cannot tell whether "compile last version of sqlite3" means the same thing, with respect to those requirements, as "build with same feature set as was used for the 'stock' replaced libsqlite3.so.0.8.6". There are tools one might use to ascertain whether at least the same entry points exist, but that would not determine whether they behave the same way. There is a SQLite pragma for ascertaining the compiled-in feature set that can be used for behavior (modulo bugs.)

If you (the OP) are lucky, either those two action statements do mean the same thing or the applications you use happen to rely on only link against a common subset of the before/after entry points and associated behaviors. I would hope, if I were to perform that replacement on any system used for important and evolving purposes, that none of the important ones relied on a now-missing entry point or a superset of the SQL accepted by the old DLL and not the new DLL. To go beyond such hope, to reasonable assurance, will take some more steps in that recipe.

Good luck!