SQLite Forum

Novice question about running UTF-8 on CLI from file.
Login

Novice question about running UTF-8 on CLI from file.

(1) By Gary (1codedebugger) on 2021-03-27 04:02:48

I'm using version 3.34.0 at the CLI and have a Windows Notepad file that has UTF-8 characters that I want to run using `.read filename.sql`.

When I try it, it fails and states "Error near ?select" which is the very first word in the file.

If I type "Pragma encoding;", it returns UTF-8.

Would you please tell me if this should work and, if so, what needs to be done?

Thank you.

(2) By Keith Medcalf (kmedcalf) on 2021-03-27 04:22:34 in reply to 1 [link]

Save the file without the BOM.

`Pragma encoding` controls the internal text storage encoding and has nothing to do with anything external to the SQLite3 library code itself.

(3) By Gary (1codedebugger) on 2021-03-27 04:47:54 in reply to 2 [link]

Thanks. It appears that cannot be done in Notepad unless using Windows 10. I'm not up to date.  Guess a different editor is required.

(4) By Ryan Smith (cuz) on 2021-03-27 11:19:16 in reply to 3 [link]

I didn't even know it can be done in Notepad in Windows 10!

Actually, I just checked, I still don't know how.
Mind sharing me how to do so in Notepad in Win10?

(I know there are other options, but I love Notepad, it's light and minimalist, one step removed from a terminal, but also lacking in features. It's so very unlike the rest of Windows.)

(5) By TripeHound on 2021-03-27 14:49:22 in reply to 4 [link]

In my version (Windows 10 Pro, 64 bit if it matters) the "Save As" dialog has an "Encoding" drop-down which includes "UTF-8" and "UTF-8 with BOM". I'm making a (perhaps unwarranted) assumption that the former does not include a BOM. I would try first with a new, empty file: it's conceivable that it won't _remove_ a BOM that's already there, although I suspect it might.

(6) By Trudge on 2021-03-27 16:24:28 in reply to 3 [link]

I program on a Mac and use BBEdit exclusively for programming. It is available for Windows.
<a href="https://www.barebones.com/support/bbedit/notes-13.5.4.html">13.5.4 Release Notes</a>

(7) By Harald Hanche-Olsen (hanche) on 2021-03-27 17:40:59 in reply to 6 [link]

> It is available for Windows.

Are you sure? It does not look that way to me.

But a bit of googling reveals this list of text editors, at least some of which run on Windows, and several of them free:
[13 Best Text Editors to Speed up Your Workflow in 2021](https://kinsta.com/blog/best-text-editors/)

(8) By Gary (1codedebugger) on 2021-03-27 17:47:38 in reply to 4 [link]

I was referring to what I read at the first link below; but I have subsequently checked on a Windows 10 Pro machine to which I have access and it is as was already stated by TripeHound.

The second link claims it can be changed by running a script but I did not try that out of fear I'd irreversibly mess something else up in the OS doing so.

https://www.bleepingcomputer.com/news/microsoft/windows-10-notepad-is-getting-better-utf-8-encoding-support/

https://www.tenforums.com/software-apps/110704-unreadable-non-ansi-characters-notepad.html

(9) By Simon Slavin (slavin) on 2021-03-28 12:14:20 in reply to 5 [link]

Notepad++ version 7.0 and above for Windows explicitly supports UTF-8 with and without BOM.

<https://notepad-plus-plus.org/resources/>

(10) By jose isaias cabrera (jicman) on 2021-03-29 13:09:56 in reply to 4 [link]

>(..., but I love Notepad, it's light and minimalist, one step removed from a terminal, but also lacking in features. It's so very unlike the rest of Windows.)

I don't care what anybody says, but that statement above is funny. :-)

(11) By anonymous on 2021-03-29 15:21:53 in reply to 7 [link]

Yes, you're right. My bad. Not sure why I thought that.

(12) By Gary (1codedebugger) on 2021-09-17 05:51:25 in reply to 4 [link]

I realize this is a bit old now but I think I understand what you mean about Notepad.  I switched to Linux shortly after this post and use Kate text editor.  I also use an older machine that runs Windows 7 and found out a couple days ago that Kate works on Windows.  I remembered this exchange and thought it might be useful.  I didn't get it from the MS store but their nightly builds page.  If you turn off most of the features, it works much like Notepad but with a long undo/redo chain, tabs for each open file, and you can save all the open tabs as a session to be opened together next time.

(13) By anonymous on 2021-09-17 10:31:34 in reply to 12 [link]

https://www.scintilla.org/SciTE.html
(The editor used for testing the Scintilla editor component)

Scintilla is also the basis for Notepad++
Multiple tabs
Sessions
Lightweight
Has a single executable version also
Portable
Supports many languages, syntax highlighting etc.
Very customizable
Fast
etc.

I have set it up to run the CLI SQLite.exe by selecting text and pressing a shortcut, results/output then goes to the output pane. Love it. I can post the SQLite/SciTE config here if others want to try it out.

Philip