SQLite User Forum

shell: three patches to also use XDG_STATE_HOME
Login

shell: three patches to also use XDG_STATE_HOME

(1) By Paul Bolle (pebolle) on 2025-06-10 16:08:44 [link] [source]

In replies to this post I hope to add three patches. The goals of those patches is to add support for XDG_STATE_HOME for sqlite3's shell. Ie, not have .sqlite_history end up in $HOME.

Patch 1: "sqlite3.1: mention ~/.local/config/sqlite3/sqliterc". A trivial update to man(1) sqlite3.

Patch 2: "sqlite3.1: document the history file". An attempt to document the history file.

Patch 3: "Use XDG_STATE_HOME for the history file". The meat of this short series.

I welcome any feedback, even on my abuse of the English language.

Licensing: I honestly do not think the patches clear the bar for copyright. But if they somehow do, the license of the patches matches sqlite's copyright policy: Public Domain.

Should compile on current development version. Tested on v3.47.2.

(2) By Paul Bolle (pebolle) on 2025-06-10 17:05:34 in reply to 1 [link] [source]

From a22ba863f8d71399ede90d8cc193765b1d8556fd Mon Sep 17 00:00:00 2001
From: Paul Bolle <pebolle@tiscali.nl>
Date: Sun, 8 Jun 2025 13:08:07 +0200
Subject: [PATCH] sqlite3.1: mention ~/.local/config/sqlite3/sqliterc

~/.local/config/sqlite3/sqliterc is one of the three possible places to
look for an initialization file. It isn't documented. Do so.
---
 sqlite3.1 | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/sqlite3.1 b/sqlite3.1
index 08b1ff262b..d47bcd76aa 100644
--- a/sqlite3.1
+++ b/sqlite3.1
@@ -140,6 +140,8 @@ continue prompt = "   ...> "
 o If the file
 .B ${XDG_CONFIG_HOME}/sqlite3/sqliterc
 or
+.B ~/.local/config/sqlite3/sqliterc
+or
 .B ~/.sqliterc
 exists, the first of those to be found is processed during startup.
 It should generally only contain meta-commands.
-- 
2.49.0

(3) By Paul Bolle (pebolle) on 2025-06-10 17:06:28 in reply to 1 [link] [source]

From c725629cbd6587a9530bc8b41ed106c546065c1e Mon Sep 17 00:00:00 2001
From: Paul Bolle <pebolle@tiscali.nl>
Date: Sun, 8 Jun 2025 13:54:28 +0200
Subject: [PATCH] sqlite3.1: document the history file

The history file wasn't documented. Add that documentation.
---
 sqlite3.1 | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/sqlite3.1 b/sqlite3.1
index d47bcd76aa..a548b4ef9f 100644
--- a/sqlite3.1
+++ b/sqlite3.1
@@ -150,6 +150,17 @@ o If the -init option is present, the specified file is processed.
 
 o All other command line options are processed.
 
+.SH HISTORY FILE
+.B sqlite3
+may be configured to use a history file to save SQL statements and
+meta-commands entered interactively. These statements and commands can be
+retrieved, edited and reused at the main and continue prompts. If the
+environment variable
+.B SQLITE_HISTORY
+is set, it will be used as the name of the history file. If it is not set
+the file
+.B ~/.sqlite_history
+will be used as the history file.
 .SH SEE ALSO
 https://sqlite.org/cli.html
 .br
-- 
2.49.0

(4) By Paul Bolle (pebolle) on 2025-06-10 17:08:37 in reply to 1 [link] [source]

From 6d4ed0dd1198d53943aaa156b7974d396a7bd30a Mon Sep 17 00:00:00 2001
From: Paul Bolle <pebolle@tiscali.nl>
Date: Sun, 8 Jun 2025 13:59:45 +0200
Subject: [PATCH] Use XDG_STATE_HOME for the history file

The init file is, well, XDG compliant. The history file is not. So make
it use ${XDG_STATE_HOME}/sqlite_history or
~/.local/state/sqlite_history (whichever is found first).

These files are only used if they exist (ie, they have to be manually
created). Only the file pointed to ${SQLITE_HISTORY} or
~/.sqlite_history will be created on exit. Whether an XDG compliant
file should be created on exit is not (yet) clear to me.
---
 sqlite3.1      |  7 ++++++-
 src/shell.c.in | 48 ++++++++++++++++++++++++++++++++++++++++++++----
 2 files changed, 50 insertions(+), 5 deletions(-)

diff --git a/sqlite3.1 b/sqlite3.1
index a548b4ef9f..f7bd1784bd 100644
--- a/sqlite3.1
+++ b/sqlite3.1
@@ -157,8 +157,13 @@ meta-commands entered interactively. These statements and commands can be
 retrieved, edited and reused at the main and continue prompts. If the
 environment variable
 .B SQLITE_HISTORY
-is set, it will be used as the name of the history file. If it is not set
+is set, it will be used as the name of the history file. If it is not set and
 the file
+.B ${XDG_STATE_HOME}/sqlite_history
+or
+.B ~/.local/state/sqlite_history
+exists, the first of those to be found is used as the history file. If
+neither SQLITE_HISTORY is set nor a file is found the file
 .B ~/.sqlite_history
 will be used as the history file.
 .SH SEE ALSO
diff --git a/src/shell.c.in b/src/shell.c.in
index 1f8ff62070..01aff5176d 100644
--- a/src/shell.c.in
+++ b/src/shell.c.in
@@ -12657,6 +12657,41 @@ static char *find_home_dir(int clearFlag){
   return home_dir;
 }
 
+/*
+** On non-Windows platforms, look for $XDG_STATE_HOME.
+** If ${XDG_STATE_HOME}/sqlite_history is found, return
+** the path to it.  If there is no $(XDG_STATE_HOME) then
+** look for ${HOME}/.local/state/sqlite_history and if found
+** return that.  If none of these are found, return 0.
+**
+** The string returned is obtained from sqlite3_malloc() and
+** should be freed by the caller.
+*/
+static char *find_xdg_state(void){
+#if defined(_WIN32) || defined(WIN32) || defined(_WIN32_WCE) \
+     || defined(__RTP__) || defined(_WRS_KERNEL)
+  return 0;
+#else
+  char *zState = 0;
+  const char *zXdgHome;
+
+  zXdgHome = getenv("XDG_STATE_HOME");
+  if( zXdgHome==0 ){
+    const char *zHome = getenv("HOME");
+    if( zHome==0 ) return 0;
+    zState = sqlite3_mprintf("%s/.local/state/sqlite_history", zHome);
+  }else{
+    zState = sqlite3_mprintf("%s/sqlite_history", zXdgHome);
+  }
+  shell_check_oom(zState);
+  if( access(zState,0)!=0 ){
+    sqlite3_free(zState);
+    zState = 0;
+  }
+  return zState;
+#endif
+}
+
 /*
 ** On non-Windows platforms, look for $XDG_CONFIG_HOME.
 ** If ${XDG_CONFIG_HOME}/sqlite3/sqliterc is found, return
@@ -13518,10 +13553,15 @@ int SQLITE_CDECL wmain(int argc, wchar_t **wargv){
       zHistory = getenv("SQLITE_HISTORY");
       if( zHistory ){
         zHistory = strdup(zHistory);
-      }else if( (zHome = find_home_dir(0))!=0 ){
-        nHistory = strlen30(zHome) + 20;
-        if( (zHistory = malloc(nHistory))!=0 ){
-          sqlite3_snprintf(nHistory, zHistory,"%s/.sqlite_history", zHome);
+      }else{
+        zHistory = find_xdg_state();
+      }
+      if( zHistory==0 ){
+        if ( (zHome = find_home_dir(0))!=0 ){
+          nHistory = strlen30(zHome) + 20;
+          if( (zHistory = malloc(nHistory))!=0 ){
+            sqlite3_snprintf(nHistory, zHistory,"%s/.sqlite_history", zHome);
+          }
         }
       }
       if( zHistory ){ shell_read_history(zHistory); }
-- 
2.49.0

(5) By Stephan Beal (stephan) on 2025-06-30 15:26:22 in reply to 1 [link] [source]

In replies to this post I hope to add three patches.

With my apologies for the delayed response...

A version of this is now in the shell-xdg-vars branch. Please try that out and see if it's faithful to your patch. If it's not, please point out the discrepancies.

BTW: there's a subtle malloc()/free() mismatch in your patch for the history file, in case you're still using it locally: If $SQLITE_HISTORY or ~/.sqlite_history are used, they're allocated using malloc(), whereas find_xdg_state() uses sqlite3_malloc(). The history file name is eventually passed to free(), which will (or should) vomit when it's passed memory allocated by sqlite3_malloc() (that may depend on sqlite's allocator configuration).

Licensing: I honestly do not think the patches clear the bar for copyright. But if they somehow do, the license of the patches matches sqlite's copyright policy: Public Domain.

Unfortunately, stating such isn't legally binding everywhere (otherwise we'd probably take a lot more drive-by patches!), so we could not use your patch as-is, but we can and do sometimes use such patches as a proof of concept and reimplement them to work around the tar pit of licensing. Since the meat of your patch - find_xdg_state() - was a copy/paste/rename for the existing XDG case, we refactored that bit into a more generic routine to cover both the history file and sqliterc.

(6) By Paul Bolle (pebolle) on 2025-07-09 17:10:26 in reply to 5 [link] [source]

Thanks for the feedback. Appreciated!

A version of this is now in the shell-xdg-vars branch. Please try that out and see if it's faithful to your patch. If it's not, please point out the discrepancies.

Cool! I hope to check that soon-ish. But if you hear nothing from me I either lost interest or everything was fine. Either way, my posts accomplish what I wanted: XDG-ify the sqlite_history file!

BTW: there's a subtle malloc()/free() mismatch in your patch for the history file, in case you're still using it locally: If $SQLITE_HISTORY or ~/.sqlite_history are used, they're allocated using malloc(), whereas find_xdg_state() uses sqlite3_malloc(). The history file name is eventually passed to free(), which will (or should) vomit when it's passed memory allocated by sqlite3_malloc() (that may depend on sqlite's allocator configuration).

Bother. But exactly the thing that might happen with drive-by patches where the submitter, almost by definition, is new to the codebase.

Thanks again! You looked at my posts, thought about the problem and moved things further. I could not ask for more.

(7) By Stephan Beal (stephan) on 2025-07-09 17:19:21 in reply to 6 [link] [source]

Either way, my posts accomplish what I wanted: XDG-ify the sqlite_history file!

Yes, but we need you (someone who uses XDG-ish things) to confirm whether we (who don't really know what an XDG is) have done it properly :).

(8.1) By Paul Bolle (pebolle) on 2025-07-21 17:53:44 edited from 8.0 in reply to 7 [source]

You've added four check-ins to trunk, so this might mean you are already comfortable with your changes. Anyhow, looking at the code and fiddling with it locally (just the history file!) revealed no issues.

A small nit: I did notice that check-in ebb346c5aa ("Correct ~/.local/config/... to ~/.config/... [...]") missed one reference to ~/.local/config/ in sqlite3.1.

(A pedant might argue that find_xdg_file() should check for empty XDG_* variables, as in

if( zXdgDir && *zXdgDir ){
The XDG_* spec mention this corner case but I'm not sure whether that extra check is important.)

Thanks for looking into this and for reimplementing my patches. It took only a few weeks for check-ins to land in trunk. This is much quicker than what can be expected from other projects. I'm impressed!

(9) By Stephan Beal (stephan) on 2025-07-21 18:01:17 in reply to 8.0 [link] [source]

so this might mean you are already comfortable with your changes.

Yup :).

missed one reference to ~/.local/config/ in sqlite3.1.

That's fixed now, thank you.

A pedant might argue that find_xdg_file() should check for empty XDG... but I'm not sure whether that extra check is important

Fair point. My current leaning is to leave it until/unless that distinction proves significant.