SQLite Forum


50 most recent forum posts by user larrybr

21:12 Reply: Существует ли справочник синтетических ошибок?Где искать исправления error near? (artifact: 1a15ca14d4 user: larrybr)

If you want help with that error, you will need to provide the sqlite3 shell input which induced the error.

And please edit the thread title so that it might prove useful someday to a larger set of readers.

17:45 Reply: Feature Request: ATTACH DB readonly? (artifact: 1c87db1032 user: larrybr)

See URI query parameters, then use the URI file specification syntax for your ATTACH. (A compile-time option to enable URIs may be necessary, depending on where you get SQLite bits.)

17:18 Reply: How to free max heap size on execution of sqlite query using sqlite3_exec? (artifact: f9d9efd9c4 user: larrybr)

Just I want to know why heap size increase on query execution? If any body technically explain it will be better. Is there method to free it?

See Ryan's post (#11) and read the doc he linked. Your questions are answered there, so there is no reason to write the same out here.

21:08 Reply: Possible bug: Unable to find an entry point (artifact: 34307dc089 user: larrybr)

... the following error is encountered on start: Unable to find an entry point named 'SIa069da76968b7553' in shared library 'SQLite.Interop.dll'

I am highly skeptical that a name like that is supposed to be exported by that DLL. I've seen many name decoration schemes, but never one that appears to encode a 64-bit number with a prefix. And all the names exported by SQLite.Interop.dll for x86 and amd64 platforms are more ordinary, semi-meaningful-to-humans identifiers.

Do you have any reason, beyond this error message, to believe that the System.Data.SQLite.Core library is, in fact, linked (albeit dynamically) to such an entry point?

19:43 Reply: Sir Richard (artifact: fba533b3c7 user: larrybr)

Please try to use thread titles which reflect the topic they introduce.

Both the expression syntax diagram and the operator table show IS in the role of a binary operator. Hence, taking those data as enabling, <expr> IS NULL clearly is permitted because NULL itself is an expression.

18:11 Reply: How to free max heap size on execution of sqlite query using sqlite3_exec? (artifact: a637fae858 user: larrybr)

How you can tell this is memory leak?

In just the code you posted, I see one likely memory leak. If errMsg is set by that sqlite3_exec() call, it is never freed by anything you show.

I am simply run sqlite query using sqlite3_exec() in c.

I hope you get understand the example std::stringstream sql;

I have written enough C++ to know that there is no need to extract a std::string from a std::stringstream when its only use is to get its content via c_str(). This cavalier approach to allocation makes me suspect those parts of your code that are not shown.

[big glop of code ...]

I'm not tempted to try to run that code fragment because I would have to create a lot of scaffolding around it; it's duplicative; it relies on a schema I am too lazy to infer; and I do not yet see any evidence or substantiated claim that an actual memory leak is occurring. What do sqlite3_memory_used() returns versus number of queries run look like? Does that function resemble a line with non-zero slope as the program keeps running?

16:46 Reply: How to free max heap size on execution of sqlite query using sqlite3_exec? (artifact: ab3919ccc7 user: larrybr)

To Gunter's question, "Have you checked to see if each execution of the query adds about the same amount to heap memory?", answer was:

Depending on number of records it increases

In other words, "No, allocated memory does not appear to grow approximately linearly as a query is repeated."

You have not described a memory "leak". Nor have you shown in what manner memory allocation increases over time. Hence, nobody can distinguish between something abnormal or objectionable and expectable, ordinary allocation growth toward a semi-stable state.

To Gunter's question, "Can you provide a minimal example (schema and queries in plain SQL) that illustrates the problem?", answer was:

I must have to check this thing in my running application on same device

Until you provide enough information for somebody else to observe your (not quite) reported problem, the assumption by others will be that what you see is a feature of your application rather than a deficiency of the SQLite library.

You may find this doc on SQLite memory allocation useful, particularly the configuration and debug tips.

02:33 Reply: IS NULL optimization on NOT NULL constraints breaks on corrupted databases (artifact: b90de8168b user: larrybr)

You make a good point regarding the spirit versus ostensible thrust of the OP's original post. I have to admit being unsure what its point really wasa, and latching onto its most obvious aspect.

However, taking "revised schema only" as a stand-in for "data altered to no longer be consistent with schema", my point about the discrepancy remains: A corrupt DB, whether made corrupt by simple, repeatable or mysterious means, is not going to have defined behavior or even behavior that should be expected to remain the same across SQLite releases.

There have been many recent changes to how the library detects and responds to DB corruption. They help avoid (further) data loss and ease the task of avoiding/reducing undefined behavior in the C/C++ UB sense. This kind of evolution simply cannot be confined to perpetuate the behavior (or misbehavior) of past releases under such conditions.

Your point about sanity checks versus performance is well taken.

a. During my perplexity on the point, I've been tempted to parody it, along lines of" "I crossed out 'Apples' on this bag of fruit and wrote 'Oranges', then pulled an apple out." But it was too crude and not quite on point. And likely to seem unkind.

17:00 Reply: IS NULL optimization on NOT NULL constraints breaks on corrupted databases (artifact: 9f5da01ff4 user: larrybr)

[Regarding effects of changing sqlite_schema table via UPDATE] ...
Which isn't obvious, and a change of behavior with respect to older SQLite versions

The effect of making your own changes to the schema table is undocumented and undefined, and always has been. Changes in undocumented behavior are sometimes intentionally avoided as the project evolves, but not always and not in any way that users should rely upon.

Your effort to inform about this particular change of behavior is appreciated. You may consider it a warning about reliance upon undocumented behavior.

16:57 Reply: Hot backup database in WAL mode by coping (artifact: 3a151087f5 user: larrybr)

... needing twice the disk space of the SQLite database.

I do not see why anybody should believe that. Your "hot backup database" is going to consume at least the same disk space as a VACUUM INTO target database, and probably more. So where do you get 2 x? I get "1 x or less".

16:33 Edit reply: Possible data race in os.unix.c:5805:unixTempFileDir (artifact: c660666202 user: larrybr)

(Edited to correct hazard preconditions and reflect fix status.)

This is, technically, a race as claimed. The only potential harm would be when:
Multiple threads initially request the temp directory name at (nearly) the same time in the same process;
For different connections;
On a build with 64-bit pointers;
Running on a 32-bit machine.

This is extremely unlikely to ever happen, of course. Nevertheless, a "cure" has been checked into the source (thanks to Dan.)

14:03 Reply: Possible data race in os.unix.c:5805:unixTempFileDir (artifact: fe55aff7f9 user: larrybr)

This is, technically, a race as claimed. The only potential harm would be when:
Multiple threads request the temp directory name at (nearly) the same time;
On a build with 64-bit pointers;
Running on a 32-bit machine.

This is extremely unlikely to ever happen, of course. Nevertheless, a "cure" will soon appear in the source.

21:46 Reply: PRINTF - newlines, tabs etc? (artifact: 0f2e119817 user: larrybr)

The printf() function does not interpret backslash-escape sequences in either C or SQLite. That is something that happens during translation, by a C or C++ compiler, from string literals in the source text to char sequences in the compiled output. And since SQLite does not implement the non-SQL convention of backslash escaping in its string literals, your examples would not work even if quoted correctly.

In SQL, the doublequote is used to delimit identifiers, not string literals. In SQL, a string literal is delimited by singlequote characters, except where they are paired within the quoted text.

15:40 Reply: sqlite3.dll is not returning control to my program (artifact: 786206c9a7 user: larrybr)

I have this in ~/.bash_aliases:

alias mdquote="perl -p -e 's/([\\[\\]\\\\\\<\\>*#])/\\\\\$1/g;'"

What you see posted is a misbegotten mishmash of that filter's output and some incremental back-edits.

15:40 Delete reply: sqlite3.dll is not returning control to my program (artifact: 46c3d39789 user: larrybr)
15:39 Reply: sqlite3.dll is not returning control to my program (artifact: 3669b7a22c user: larrybr)

I have this in ~/.bash_aliases:

alias mdquote="perl -p -e 's/([\\[\\]\\\\\\<\\>*#])/\\\\\$1/g;'"

What you see posted is a misbegotten mishmash of that filter's output and some incremental back-edits.

23:55 Reply: sqlite3.dll is not returning control to my program (artifact: 65a819ce49 user: larrybr)

I do not see anything definitely wrong with what you have shown. This makes me think something you're doing not yet shown is wrong. It would be a tedious guessing game to venture all the ways you might be erring.

Instead, I created this silly little C++ish program: #include <iostream> #include <string> #include <assert.h> #include "sqlite3.h" // using std::cout; using std::cerr; using std::endl; using std::string; // int execCallback(void*,int nc,char** pzColValues, char** pzColNames){ cout << "Result row:" << endl; for( int ic = 0; ic < nc; ++ic ){ char * zValue = (pzColValues[ic])? pzColValues[ic] : "NULL"; cout << " " << pzColNames[ic] << ": " << zValue << endl; } return 0; } // int main(int an, char *ap[]){ if( an < 2 ){ cerr << "Usage: sdb_futz <dbName> [<query1> ...]" << endl; return 1; } string dbName(ap[1]); sqlite3 * db = 0; int rc = sqlite3_open( dbName.c_str(), &db ); if( rc != SQLITE_OK ){ cerr << "Cannot open " << dbName << " as DB." << endl; return 2; } int nFlubs = 0; for( int na = 2; na < an; ++na ){ char * zErr = 0; string query(ap[na]); cout << "Running query: " << query << endl; rc = sqlite3_exec(db, query.c_str(), execCallback, 0, & zErr); if( rc != SQLITE_OK ){ cerr << "Exec failed with error " << rc << " (" << (zErr? zErr : "?") << ")" << endl; ++nFlubs; sqlite3_free(zErr); } } rc = sqlite3_close(db); return ( rc == SQLITE_OK && nFlubs == 0 )? 0 : 2+nFlubs; } // . This is what I did to build it: REM Prior to this, get 64-bit sqlite3.dll, sqlite3.def and REM sqlite3.h downloaded and extracted to this directory. REM Then, in a shell setup for MSVC 64-bit build tools, run: lib /def:sqlite3.def /out:sqlite3.lib /machine:amd64 REM Next, create something like above source: emacs_edit sdb_futz.cpp REM Then build and run it: cl -DDEBUG -EHsc sdb_futz.cpp /link sqlite3.lib sdb_futz furd.sdb "create table Silly(why text)" sdb_futz furd.sdb "insert into Silly values ('No reason'),('Why am I silly?')" sdb_futz furd.sdb "select * from Silly" "select count(*) from Silly" .

This ran as I would expect, so I'll not bore anybody with output.

I propose that you figure out how your code and build differ from the above. You can replicate my build, then use DUMPBIN to examine our objects and image to perhaps discover if linkage or calling convention has gone awry. (Naming conventions should prevent such problems, but they may have been circumvented.)

Your observation, "problem temporarily goes away if I restart VS2019", is not much to go on. But it does suggest you are causing the dreaded C/C++ undefined behavior to occur, just as your other symptoms suggest. (Failure of return from a SQLite API means something fundamental is wrong.)

You may notice that my build does virtually nothing to establish DLL linkage or set calling conventions. You may want to try that among your other explorations.

If you are serious about getting help with your problem here, you will need to simplify your code and build to something others here can reasonably hope to replicate. The Visual Studio IDE is nice, but it hides a tremendous amount of detail in its project files.

Obviously, you are building something which plays poorly with the sqlite3.dll published by the SQLite project. Yet, as you can see, I can build something that plays well with that DLL with little trouble.

01:15 Reply: Hot backup database in WAL mode by coping (artifact: 095fb5be24 user: larrybr)

Those 2 statements are executed one after the other by the CLI shell.

01:54 Reply: Usage of application_id and magic.txt (artifact: 660f667ab9 user: larrybr)

[On magic numbers listed in <SQLite-source-root>/src/magic.txt ...]

This is still recommended path forward?

It is recommended if you want "utilities such as file(1)" to report something other than "SQLite3 Database" when given your particular DB files. It is up to you to determine whether to go to this trouble. To assess that, you need to consider how the "magic" text will be made to appear in the /usr/share/misc/magic configuration file used by the "file" utility on systems where your particular DB files are likely to appear and have provenance unknown to those who actually need to discover it. Given those predicates, an unqualified "recommended" seems a bit much in most cases.

If we're intending to do things the "Right Way" (TM) ...

(Chuckle) In my opinion, you or others in your organization are far better poised to decide what "the Right Way" is.

19:21 Reply: Minor typo in copyright statement (artifact: 5febca3ca1 user: larrybr)

Fixed (but not yet published.) Thanks.

16:59 Reply: CLI's .dump and ASCII vs UTF8 (artifact: ac7cda0711 user: larrybr)

This has been fixed

I don't see it on the live site, it's probably on Draft only. I didn't check the timeline.

It's in the docs repository, to be published in due course. (Minor changes are not normally published immediately.)

I'm not sure how much should be said about this

Well, given the above, the encoding of the .dump output in not even UTF, it is whatever the text values contain...
Because again, UTF-ness is not enforced in SQLite. So it's even more ambiguous than ASCII or UTF-8.

The .dump output data are returns from sqlite3_column_text(...), which are nominally UTF-8 for differing overall DB text encodings. Here, "nominally" means when text inputs have consisted of valid code point sequences. For DB encodings other than SQLITE_UTF8, those returns will be translated from the raw stored content.

I don't think it should be .dump's job (or option) to somehow filter invalid UTF-8 text that has been stored by users. It's job is simple, restricted to producing a text representation of the existing DB.

Also there won't be a BOM if uses non-ASCII multi-byte UTF-8 codepoint.

I suppose a --bom option might make sense for the .dump command. However, given that its output will normally be either subject to much further processing or fed directly to sqlite3, I do not yet see real value or necessity for that.

[general point that invalid code point treatment should be documented]

The fact that SQLite stores text as given to it, with possible translation between encoding schemes but no rejection of technically invalid code points, deserves mention somewhere. My hesitation is against cluttering the docs with this level of detail wherever the terms "UTF-8" or "UTF-16*" are used. I will find a place to mention this behavior, and maybe link to it where use of those terms might lead to false expectations.

... there should be a pragma to check basic UTF-8 (and UTF-16) of text values, which ...
I'm merely thinking aloud, and I'm used to being ignored on this forum by now.

This deserves a separate thread, and attention for a post-3.37 release.

14:07 Reply: CLI's .dump and ASCII vs UTF8 (artifact: 040cfadf44 user: larrybr)

[The .dump doc] says ASCII, but .dump seems to produce UTF instead.

This has been fixed. Thanks.

[I]t might be nice to have a basic QC check pragma for UTF*-ness* of all text-typed values in a DB, BTW.

Yes, it might. This is among the arguments to be made for shell extensibility, IMO.

More details on how .dump handles international characters might be nice to have in the doc. My $0.02 :)

For most platforms, .dump output is fprintf() output. I think little needs to be said about that since it is either uninteresting or strictly a platform quirk.

For Windows, .dump output is little more than fprintf() unless output is going to the console. In that case, (as a special dispensation to that quirky but common destination), output is translated as needed to MBCS because that console handles UTF-8 poorly. I'm not sure how much should be said about this, if anything, because it will rarely apply to output used programmatically.

12:48 Edit reply: CLI's .dump and ASCII vs UTF8 (artifact: a7bd2d67a2 user: larrybr)

You have made a convincing case that said doc should say "UTF-8" rather than "ASCII". Please clarify if there is some additional problem brought to light by your post. (I gather 1 error and 1 feature suggestion from it.)

BTW, ASCII is strictly a 7-bit coding scheme, so "°" cannot be encoded in it with any small number of bytes.

12:45 Reply: CLI's .dump and ASCII vs UTF8 (artifact: cda38384b1 user: larrybr)

You have made a convincing case that said doc should say "UTF-8" rather than "ASCII". Please clarify if there is some additional problem brought to light by your post.

BTW, ASCII is strictly a 7-bit coding scheme, so "°" cannot be encoded in it with any small number of bytes.

18:58 Reply: Delete duplicate rows with an extra character (artifact: 4f5c909ff8 user: larrybr)

Here is how to get the rowid for delete candidates: SELECT x.rowid FROM players x, players y WHERE substr(x.player_tag,2)=y.player_tag AND substr(x.player_tag,1,1) IN ('#'); . Calling that query "Extras", the delete is: DELETE FROM players WHERE rowid IN (Extras); . It helps here that each row has a unique rowid.

23:10 Reply: Best way check if a file is a SQLite DB? (artifact: 142f659d06 user: larrybr)

The SQLite 3 database header is stable; it will not change for any major version 3 of the library.

13:58 Reply: Insert operation leads to exception on Windows when installed using msi (artifact: 844b701257 user: larrybr)

You need to arrange that, once installed, your application can be used by those for whom it is intended to be used. This means that the DB file will need to have permissions consistent with modification by those users.

How to achieve that using your preferred installer software is off-topic here. Whether the solution is to use installer features or some kind of post-install fixup script, discussion of it should happen elsewhere. I expect there are fora dedicated to your installer where your problem would be readily solved and topical.

You might also consider having the application itself create the DB file when it does not yet exist.

16:53 Reply: Question about memory management (artifact: d8522ee379 user: larrybr)

Unless you are using an in-memory database, doing a CSV import should not present any particular problem for the CLI.

Setting SQLITE_DEFAULT_MEMSTATUS to 0 will hinder getting memory usage data from the SQLite instance, but will not be the culprit in excessive memory use. If you want to see where that problem is, you may want to leave that option at its default (1).

I think you should explain how your Clarion code does the import before hoping for much help here. SQLite is not generally a resource hog.

15:40 Reply: sqlite3ext.h missing "/" in comment "* Version 3.37.0 and later */" near line 656 (artifact: 07765b0cc2 user: larrybr)

Thanks for the tip. Fixed now.

18:51 Reply: Get SQLITE_READONLY when using UPDATE command (artifact: 528f7d10a6 user: larrybr)

The "proper" URL would be "file://Users/David/test.db" which would be the file \Users\David\test.db on the current drive (ie, the current directory location when the program is loaded.

The format specified is FUBAR.

SQLite does not require the double-slash between the URI's scheme part and the remainder. However, per my reading of RFC 3986, it should.1 So, in that sense, the OP's URL is improper and other adjectives may apply.

I'm not sure how this deviation arose or should be handled now given the backwards compatibility issues.

  1. This deviation creates an ambiguity between absolute and relative pathnames. SQLite interprets a leading/only slash after the scheme as the beginning of an absolute path. To get a relative path, no leading slash should be used.
18:23 Reply: Get SQLITE_READONLY when using UPDATE command (artifact: 1ed1a9bd37 user: larrybr)

Interesting. I expect that was the Unix-experienced faction's work.

18:22 Reply: Get SQLITE_READONLY when using UPDATE command (artifact: 33e8963259 user: larrybr)

I pretty much agree, except for:

The filename you used, unless URI filenames are enabled, would name the "database" as the alternate data stream called ...

I don't think that stream names (whether alternate or not) can contain path separators. I've not tested this, but it would be pernicious if permitted.

18:18 Reply: Get SQLITE_READONLY when using UPDATE command (artifact: 1de5858aa6 user: larrybr)

Now I get a hex E from result.

You mean you get an error:


That would be "hex F".

Try specifying a proper filename in proper filename format and see if that helps (ie, forget about using the URI format, use the CPM/DOS/Windows filename format standard that has existed unchanged since at least 1979 if not earlier.

The OP's' filename will be fine when either stripped of its leading "file:" or when passed with the SQLITE_URI flag in sqlite3_open_v2().

The '\'-as-path-separator convention was misbegotten from the beginning. It is no longer necessary to use it instead of '/' for Win32 APIs, and has not been since Win16 was obsoleted nearly 2 decades ago. (FWIW, the backslash convention was hotly contested with Microsoft, but imposed by IBM as MS-DOS was made ready for use in PC's, evolved from Tim Paterson's 86-DOS.)

16:07 Reply: Get SQLITE_READONLY when using UPDATE command (artifact: bf78d1f0cb user: larrybr)

(Quoting code fragments as context for response fragments:)

database.cpp file:(in constructor)
m_Main_Db_File = "file:\\Users\\David\\test.db";

It appears that you intend to use a URI database specification. This one is in good order. I prefer (single) forward slashes, as Windows has accepted them since the 16-bit codebase became history.

// Make the sqlite3 temp directory.
LPCWSTR zPath = ApplicationData::Current().TemporaryFolder().Path().data();
char zPathBuf[MAX_PATH + 1];
memset(zPathBuf, 0, sizeof(zPathBuf));
WideCharToMultiByte(CP_UTF8, 0, zPath, -1, zPathBuf, sizeof(zPathBuf), NULL, NULL);
sqlite3_temp_directory = sqlite3_mprintf("%s", zPathBuf);

If your DB file will be in a writable directory, there is no need (evident here to me) for setting a different temp directory. It's a complication at best.

int result = sqlite3_open(m_Main_Db_File, &ptrMain_Db);

You need to use sqlite3_open_v2() with a SQLITE_OPEN_URI flag.

I moved the database file to my user folder which I should have full permission rights to. Now I get a hex E from result.

Getting a different result should be considered progress. That return is also known as SQLITE_CANTOPEN. As the linked doc indicates, the open failure could be either the main DB file or one of the potential auxiliary files. Eliminating the sqlite3_temp_directory set would help distinguish cases here.

Without the file:, single back slashes which Windows did not like, and forward slashes.

It is not Windows which disliked single backslashes. (It never saw them.) You should study what C/C++ string literal escaping conventions mean and do. A double-backslash becomes a single backslash in the compiled literal.

This is different from when the database was in my project folder.

Yes, and with a little more information that will be a potent clue, I think.

At this point, I suspect your problem is that Windows, when given file:\Users\David\test.db as a pathname, refuses to create a file object under that name. Telling SQLite to treat that text as a URI will keep the "file:" part from being passed to the OS file open API.

In VS I have installed SQLite/SQL Server Compact Toolbox. From this too I can always access and modify any SQLite database. This is how I have been making changes to the database so far. Is this tool causing my problems?

I was unaware that the tool could deal with SQLite databases. (I'm skeptical that it can.) But if you are not keeping a SQLite database open in that tool, I highly doubt it will lead to the problem(s) you see now.

15:18 Reply: Is the order conserved in a table from a VALUES clause? (artifact: 2c57e13d27 user: larrybr)

I almost hate to spoil the fun here, but ...

Thanks for your input Larry - I totally get that relying on undocumented behaviour is not a good strategy!

The thrust of my argument now is that the current behaviour should be fixed and documented.

Yes, I understood that.

What is documented is that an ORDER clause enforces some kind of ordering. If feasible, it will be optimized away where the implementation naturally (in its current evolution) achieves the specified ordering. You can rely on that.

The extra guarantee you seek/suggest is highly unlikely to be made by the present SQLite project. In part this is because VDBE code generation is complex enough without imposing requirements which are contrary to long-standing SQL convention. And, I venture, further cause to decline your invitation is that it would weaken the message: If you want ordering, use ORDER BY. The docs have been quite clear that, absent an ORDER BY clause, no ordering is guaranteed. I do not foresee an ever-growing list of exceptions appearing with those warnings.

This stance on ordering is especially important for input to the windowing functions, where ordering (or misordering) is baked into the result rather than merely affecting result ordering.

23:13 Reply: Get SQLITE_READONLY when using UPDATE command (artifact: a691e55d81 user: larrybr)

I think it's time to show your sqlite3_open() call.

It would help rule out a whole category of problems if you were to temporarily substitute for your DB filespec a pathname which you are quite certain can be written by whatever user is getting your unexpected SQLITE_READONLY return. If that continues through the substitution, we can focus on your call. Otherwise, the investigation needs to examine file permissions.

21:23 Reply: sqlite3_threadsafe() return value (artifact: b1f79e2c43 user: larrybr)

This should be clearer. I chose less verbiage than you or Keith suggest because the immediate context would render it surplusage. Please feel welcome to voice reasoned disagreement with this opinion.

20:12 Reply: sqlite3_threadsafe() return value (artifact: 2cca976410 user: larrybr)

I read the allegedly incorrect doc quote, saying "The sqlite3_threadsafe() interface predates the multi-thread mode and start-time and run-time mode selection and so is unable to distinguish between multi-thread and serialized mode ...", as entirely consistent with the fact that it returns a constant set at compile time. The actual running mode, as set at runtime via pragma and possibly limited by compile-time preprocessor variables, can vary for a given instance of the library code, whereas the return from sqlite3_threadsafe() cannot.

I am inclined to say there is no doc bug here. However, the evidence suggests a better wording might be be clearer to you and perhaps others. Would you care to suggest something specific?

17:54 Reply: Is the order conserved in a table from a VALUES clause? (artifact: ed4938d840 user: larrybr)

You are clear that this may change in the future? However, I can rely on this functionality for versions up till now?

You can rely on currently released versions to continue their present behavior. (There are no manipulations of history in the source archives.)

Anybody is welcome to rely on undocumented, implementation-defined behavior. They do so at their peril with respect to continuation of such behavior in future or other as-yet-untested releases.

17:28 Reply: v3.36.0 fails to compile under Ubuntu (artifact: 7f36fcaa10 user: larrybr)

Dan Kennedy has fixed this build problem with this checkin.

00:53 Reply: sqlar : how to remove files (artifact: 68d015f77d user: larrybr)

Perhaps you or others interested can try breaking the archive_remove branch tip. Its name should be archive_remove_and_glob now. The archive content is subject to glob matching for --remove, --list and --extract subcommands. Glob matching for incoming files (--insert and --update) may come later.

23:25 Reply: Unable to open the connection (artifact: e3192cf089 user: larrybr)

I would like to keep the DB created outside the application context @ c: drive so that multiple process can access it .

I do not understand this "would like". To me, it is as if you said, "I would like the database to exist in a file so that more than one process can access it." To that, the answer is: "That's what happens when you name a file to the open API." This is so obvious that I think you must mean something else, but I cannot fathom what that is.

Able to create the connection and get connection object but unable to open connection.

It would help if the meaning of "create the connection", "get the connection object" and "open connection" did not have to be guessed. What APIs are you calling, in what order, and with what effect?

I tried read/write access to that drive.

What does "drive" access have to do with this? Can we restrict discussion to a specific file that you name as the database to an SQLite API that expects one?

18:59 Reply: sqlar : how to remove files (artifact: 03ba9de2ef user: larrybr)

I considered using glob(x,y), and may have it in before the merge to trunk. Because feature freeze is imminent, I elected to defer that nicety in favor of leveraging existing code to process a known file list, without changing it. I expect to get globbing in later. Whether I can get it working and properly tested before v3.37 feature freeze is uncertain, so I want to be sure to get the basic function in first.

17:53 Reply: sqlar : how to remove files (artifact: 42ef352bd1 user: larrybr)

Perhaps that feature should be backported.

The feature is going into the 3.37 release.

14:02 Reply: Request: Allow VACUUM INTO with a new page size even in WAL mode (artifact: 924c0a71e2 user: larrybr)

Richard has implemented your suggested behavior, in a similar manner.

13:05 Reply: Insert operation leads to exception on Windows when installed using msi (artifact: 9297a20a9f user: larrybr)

Your application, (identity of which remains a mystery), is clearly doing something which requires permission(s) not given to some ordinary users.

The SQLite project does not publish any application which uses either ".msi" installation or exception-style error handling. Your issue is with the mystery application and should be taken up with its vendor, author or publisher. Or you need to install it differently. (I can only guess on this.)

12:57 Reply: Assistance with using SQLite in PAMGuard (artifact: e49140200b user: larrybr)

[lots of PAMGuard-specific moans] because database TowST could not be resolved. ...

Does this shed any light on what my issue could be?

It sheds the same light as your previous post which appeared to show that "the database" of which you spoke was not well defined. That is why I advised: you need to specify what, specifically, you wrote/did/clicked etc. to designate an actual database file for the SQLite library to open and operate upon.

A problem we often see in this forum is this: A user specifies a filename for the database file but does not know where in their filesystem(s) that file actually resides. Then they get confused because their program's effect(s) upon "the database1" cannot be observed when they use another tool to examine "the database2", which happens because database1 is a different byte sequence than database2.

Did you specify an absolute path as "the database"? If not, your problem is highly likely to be the same above-described one we see here often. Until I see an answer to this question, I will assume you are suffering that same problem. The solution is to specify an absolute path to PAMGuard for its database, then specify the same absolute path to whatever other tool you use to observe PAMGuard's output data.

19:53 Reply: --skip option does not work correctly. (artifact: 90f8d99465 user: larrybr)

Thanks for reporting this. And thanks to Harald for making the problem clearer.

This is fixed here.

13:11 Reply: Database on android SD CARD (artifact: e14107dabe user: larrybr)

The reason for not wanting to root is that I wanted to develop an application / extension that others could easily use without rooting their phone. If it were just for me, fine, but if I want others to use it who may not be as enthusiastic about rooting, wouldn't it be great if it could work?

I do not question your motivation for wanting to avoid rooting a phone. I just don't think that's a constraint that is going to be flexible during troubleshooting or allow you to do the experiments that would allow you to figure out what the relevant factors are that lead to today's problem.

When you said its a clue that the comment noexec is relevant, How can you say that when the permission says RWXRWX--- Doesn't that say that the base user should be able to execute?

I saw the difference between "RWXRWX---" and "rw rw ---" (on your "emulated storage") as significant and related to the "noexec" mounting. It suggests that the 'x' bit is less significant (with respect to today's problem) than you seem to think.

The per-file execute permission is separate from the effect of the noexec mount option. I would not expect the per-file permission bits to reflect whether the block device was mounted with the noexec option. (They do not on Linux, and I'm too lazy to check this on other Unix-like systems.) They do not mean that that, once loaded, program image text segments from the file will be physically executable. IOW, the 'x' bits do not mean quite what you think they mean.

While you are likely right that my issue is with my environment and not with SQLite, my hope is that other SQLite enthusiasts may have bumped up against this issue and be able to help with how to solve it.

I do not hope to disabuse you of your hope(s). My recommendation of a different forum was intended to improve your odds of success and help keep this forum on-topic. Other Android developers are much more likely to have the pertinent knowledge, and on an Android forum they are less likely to regard the issue as off-topic which will also improve your odds of getting a useful answer.

21:46 Reply: Database on android SD CARD (artifact: 605fa14ffe user: larrybr)

On my SD card the permissions were RWXRWX--- but on my emulated storage its rw rw ---

That's a decent clue that Stephan's "noexec" tip is pertinent.

I selected the 20 FE phone specifically for the SD card so I could do this project without rooting my phone.

The "cannot root phone" constraint is not one likely to be catered to.

I'm on the verge of giving up on SQLite even though I really really want it to work!!

If wishes were fishes, ...

What makes you think this has anything to do with SQLite, other than that it is ultimately executed (like anything else running) and your environment is blocking execution?

You should go to an Android development forum to get advice on your problem. You have a platform issue rather than a SQLite issue.

More ↓