When compiling the SQLite shell from the amalgamation with the following command using MinGW64

gcc shell.c sqlite3.c -o sqlite3.exe

compilation succeeds without warnings or errors. But running the resulting executable

d:\sqlite3470>shell
SQLite version 3.47.0 2024-10-21 16:30:22
Enter ".help" for usage hints.
d:\sqlite3470>

just shows the version and help info and then exits immediately.

I inspected the shell code (shell.c) and detected that 2 new internal shell functions were introduced in version 3.47.0, namely sqlite3_fgets and sqlite3_fputs (via ext/misc/sqlite3_stdio.c).

Both functions allocate temporary memory using the function malloc. However, this temporary memory is then freed using the function sqlite3_free. This is wrong and leads to a sort of null pointer exception, terminating the application. For freeing the temporary memory the function free should be used.

Will you do me a favor and try building with MinGW64 with this check-in: 29164601. That's the last one that I know works. Then, try the next check-in, f97f9944, and this one should fail (aborts right away). Will you let us know if this is the behavior that you see?

When compiling the SQLite shell from the amalgamation with the following command using MinGW64

We have a prior report of this in the forum but we don't have a clue as to what causes it didn't have a clue until...

Both functions allocate temporary memory using the function malloc. However, this temporary memory is then freed using the function sqlite3_free.

Nice catch! That mismatch is now patched in the trunk. Please let us know if that resolves this the problem and, if it does, we'll cherry-pick this fix back into the 3.47 branch.

Alternately, if you already know that it works with 3.47 because you've patched yours, please let us know.

Edit: we've gone ahead and cherry-picked this fix to the 3.47 branch but we don't yet know if it is what solves Jose's problem. If you're able, please try out either branch-3.47 or the trunk and let us know if the immediate-exit behavior is resolved.

Compiled in the very same environment, "sqlite3_fgets/sqlite3_fputs::_setmode" does not seem to work properly.

Disabling it is a workaround for a small charset only.

However, the provided Windows binary does work flawlessly regarding both issues – free/_setmode. How is this possible ?

Compiled in the very same environment, "sqlite3_fgets/sqlite3_fputs::_setmode" does not seem to work properly.

"Does not seem to work properly" is not enough info for us to be able to resolve that.

What, precisely, is the problem you're seeing and under which build/environment are you seeing it?

However, the provided Windows binary does work flawlessly regarding both issues – free/_setmode. How is this possible ?

That it worked properly with the mismatched malloc/free() pairs was a fluke. Depending on how the underlying allocator is set up, such a mismatch can be harmless (but is a bug-in-waiting even when it works). As to setmode: i can't even begin to speculate what differences there are between the plain-vanilla Windows API and their MinGW/Cygwin counterparts.

This patch:

--- shell.c	Mon Oct 21 17:48:06 2024
+++ shell-MinGW.c	Sun Nov 10 20:19:39 2024
@@ -462,13 +462,25 @@
     */
     wchar_t *b1 = malloc( sz*sizeof(wchar_t) );
     if( b1==0 ) return 0;
+#ifdef __MINGW32__
+    if (IsConsole(in)) {
+      DWORD NumberOfCharsRead;
+      ReadConsoleW(GetStdHandle(STD_INPUT_HANDLE), b1, sz, &NumberOfCharsRead, NULL);
+      b1[NumberOfCharsRead]=0;
+	}else{
+      _setmode(_fileno(in), _O_U8TEXT);
+#else
     _setmode(_fileno(in), IsConsole(in) ? _O_WTEXT : _O_U8TEXT);
+#endif
     if( fgetws(b1, sz/4, in)==0 ){
-      sqlite3_free(b1);
+        free(b1);
       return 0;
     }
+#ifdef __MINGW32__
+	}
+#endif
     WideCharToMultiByte(CP_UTF8, 0, b1, -1, buf, sz, 0, 0);
-    sqlite3_free(b1);
+    free(b1);
     return buf;
   }else{
     /* Reading from a file or other input source, just read bytes without
@@ -530,13 +542,22 @@
     if( b1==0 ) return 0;
     sz = MultiByteToWideChar(CP_UTF8, 0, z, sz, b1, sz);
     b1[sz] = 0;
+#ifdef __MINGW32__
+    if (IsConsole(out)){
+      DWORD NumberOfCharsWritten;
+      WriteConsoleW(GetStdHandle(STD_OUTPUT_HANDLE), b1, sz, &NumberOfCharsWritten, NULL);
+    }else{
+#endif
     _setmode(_fileno(out), _O_U8TEXT);
     if( UseBinaryWText(out) ){
       piecemealOutput(b1, sz, out);
     }else{
       fputws(b1, out);
     }
-    sqlite3_free(b1);
+#ifdef __MINGW32__
+	}
+#endif
+    free(b1);
     return 0;
   }
 }

applied to shell.c (3.47) together with "-DSQLITE_U8TEXT_ONLY -municode" compilation options seems to solve MinGW issues.

Should be tested thoroughly.

Please download the latest trunk check-in from https://sqlite.org/src/tarball/trunk/sqlite-trunk.tar.gz and try again building using your MinGW64 install. Report back whether or not you are successful.

Dr Hipp,

Is the matching amalgamation available somewhere ?

You can easily create the amalgamation yourself using either:

./configure && make sqlite3.c

Or

nmake /f Makefile.msc sqlite3.c

Not so easy : No VS/Linux available on this laptop.

But you have MinGW/MSYS. Run the "./configure && make sqlite3" using that.

No MSYS.

Dr Hipp,

Have replaced "sqlite3_stdio.c" inside "shell.c" with the latest trunk version and successfully compiled with "gcc -municode shell.c sqlite3.c -o MySqlite.exe" under W7.

Test environment:

• MinGW 7, 11, 14 (mcf-msvcrt)
• W7/32&64, W11/64
• CMD, Far Manager
• Cut&paste/keyboard combinations, redirections
• "multilang.sql" (previously provided by Larry) with FR added
• Viewers : CMD, Far Manager, Notepad++, Total Commander

Works flawlessly in all cases, except W7 + CMD/Far Manager + cut&paste + some(?) "input-characters-replaced-by-squares".

Have been unable to reproduce this issue under W11.

Thank you for testing.

If it works on Win11, I'm saying that the issue is resolved. Win7 is out-of-support by Microsoft for almost five years (ten years, if you are referring to "mainstream" support), so it seems like that is not something we should spend a lot of time on.

Wise decision.

Anyway, even under W7, there is no problem as long as input characters are properly rendered by the console.

Just out of curiosity : What about the SHELL_CIO_CHAR_SET (CIO_WIN_WC_XLATE) macro ?

SHELL_CIO_CHAR_SET is legacy cruft. It has now been removed.

Works flawlessly in all cases, except W7 + CMD/Far Manager + cut&paste + some(?) "input-characters-replaced-by-squares".

If you are going to use Win7, I would install some software with a more sophisticated terminal and do not use the Windows DOS command tool. Use something like Cygwin or MSYS2. Just a thought...

There are 2 other methods, namely sqlite3_fopen and sqlite3_popen which use malloc and free for managing temporary memory. You may consider to consequently use sqlite3_malloc and sqlite3_free (instead of malloc and free). BTW, the shell code itself also uses frequently malloc and free.

Thank you for pointing those out. Ideally they should use sqlite3_malloc/sqlite3_free(), but so long as they're using matching pairs of allocator/deallocator they're okay. The app-level code (the shell and the sqlite3_stdio helper which supports it) are not as strict as the library-level code is about which allocators they use, but we'll get those cleaned up to use the library's allocator API.

Edit: it turns out that it's not quite that straightforward in the shell. In some places it works with memory from the underlying line-editing library, and it has to use malloc/free/realloc() with that memory. It also turns out that that's the reason which sqlite3_stdio.c used malloc/free instead of sqlite3_malloc/sqlite3_free(), so that change needs to be swapped to use those allocators. (Edit: that part isn't necessary because those allocations never leave sqlite3_fgets(). i.e. it all looks okay now.)