Using android AAR with modified sqlite_open_v2() - attach gives problems
(1) By Willem (sn0wbl1nd) on 2021-01-14 04:27:33 [link]
What we did: add a callback to the sqlite3 code that allows us to do things to the sqlite db if a certain pattern matches in the file (basically custom URI). We compile this as a custom libsqliteX.so and use AAR to talk to it under android. We also attach a database to the main db file that is opened. When this attaching db is empty: no problem. When it has tables in it: everything hangs - attach never returns. All this happens before our modified sqlite3_open_v2() returns. It works gloriously on linux and android when using only native code. I realize that we probably should have gone the VFS route, but this is a product in development. We will certainly look at that in the near future. Can anyone point me to something that might be stopping us from attaching the db? Thanks! Willem
(2) By Larry Brasfield (LarryBrasfield) on 2021-01-14 14:00:14 in reply to 1 [link]
Sorry, but this has to be said: Your evidence clearly implies that some code of your own creation is causing the problem. Yet you give only the barest hint as to what the code is. [a] I could offer a zillion guesses as to what is wrong with that code if I could type faster and was immortal. That would be a waste of my time, other readers' time, and your time. So, I await a better set of clues as to what went wrong. [a. The subset of "code" that uses the callback technique to do "things" is quite large, undoubtedly super-astronomically large. ] Another point: It is considered rude to repost the same inquiry under different thread titles. If, after a period of time, no reply appears in your single thread, the better approach is to reply to your own post. You might even use that opportunity to add some details that would improve the prospects of a useful reply. However, 0.4 hours is too short a time to do either.
(3) By TripeHound on 2021-01-14 14:06:49 in reply to 1
I've virtually no experience of working with SQLite at the source level, and none with using it on Android, but with a "generic debugger's hat" on, it seems to me that one of the most likely causes (or, at least, the one to eliminate first before digging deeper) is that your custom `libsqliteX.so` has not been "built properly" in some manner. Therefore, assuming you've not already done so, I would suggest building `libsqliteX.so` with _no modifications_ to the source. If your app hangs/crashes, then it suggests the build process (or whatever is involved in getting your app to use your custom version of SQLite) is at fault. If/when an unmodified custom-build works while doing "normal SQLite things", I would then make a small change to the code to "prove beyond doubt" that it is your custom version of SQLite that's being used: perhaps write something to a known file or logcat; make one of the interface-level functions return an easy-to-see-it's-different value. It probably doesn't matter at this stage if it makes SQLite unusable: you're just proving that you can alter the code and see the effects. Once you _know_ you can customise the normal SQLite code and successfully build and use it from your app the focus switches to whether you're "breaking" SQLite and/or Android in what you're trying to do... I assume [this post by omen_wand](https://sqlite.org/forum/forumpost/2fffd3c4f9) relates to the same problem (if not, the two of you should probably collaborate!). While I don't know much about the internals of SQLite's code, a phrase from that post "_returns a handle with two attached database connections_" _feels_ like the sort of thing that would be easy to "get wrong": thinking purely off the top of my head, I can imagine that there might be things that happen later in the official `sqlite3_open_v2()` code that mean trying to do the equivalent of `ATTACH` at the point you're doing so means it's never going to work (internal structures haven't been initialised properly, that sort of thing). Again thinking in a generic-debugger manner, I'd try working though things in stages. Start with a _small_ change to `sqlite3_open_v2()`: perhaps instead of opening the passed-in database file, always open `"test.db"`, or open `"mydb-test.db"` when `"mydb.db"` was passed-in. If that works, try something a little more involved. If you get to this stage, you probably need someone who _does_ know the innards of SQLite and you probably need to show examples of the changes you're trying to make...
(5) By Willem (sn0wbl1nd) on 2021-01-14 23:59:44 in reply to 3 [link]
Thanks for your considered answer. I appreciate it. I am just posting to follow up for completeness. Yes, we did build the unmodified version first and that worked fine, confirming it is not our setup. We created a C-style function that accepts a db name, opens it, attaches a "shared" database file, installs a preupdate callback, and returns the open sqlite3* to the user. Relatively standard and works like a charm. The reason we want this is because our code needs both the "local" and the "shared" db present, and this is the method we prefer. Then we compiled sqlite into a custom library where our function masks/replaces the existing sqlite3_open() and sqlite3_open_v2() functions _if_ the filename string starts with e.g. `ourthing://`. This works very well as well. (We happened to have done this using callbacks, but the above explains the intent.) It stops working when we then involve the AAR and libsqliteX.so. Almost everything works _except_ the attach step. As was suggested: our current work-around is to skip the attach step before returning the open sqlite3*. We then use the java interface to attach the shared db, and it seems to work nicely. If we do figure out why this happens I'll make sure to post an update here. Thanks for the constructive replies. W
(4) By Keith Medcalf (kmedcalf) on 2021-01-14 14:19:59 in reply to 1 [link]
What does doing something to an "SQLITE DB" have to do with anything? > We also attach a database to the main db file that is opened. SQLite3 does not do this. You can "ATTACH" a database file to a connection via the ATTACH SQL command. This does not attach a database to the main db -- it ATTACHes an additional database file to an existing connection. You need to be more precise in your descriptions.