SQLite Forum

Novice question about running UTF-8 on CLI from file.
Login

Novice question about running UTF-8 on CLI from file.

(1) By Gary (1codedebugger) on 2021-03-27 04:02:48 [link] [source]

I'm using version 3.34.0 at the CLI and have a Windows Notepad file that has UTF-8 characters that I want to run using .read filename.sql.

When I try it, it fails and states "Error near ?select" which is the very first word in the file.

If I type "Pragma encoding;", it returns UTF-8.

Would you please tell me if this should work and, if so, what needs to be done?

Thank you.

(2) By Keith Medcalf (kmedcalf) on 2021-03-27 04:22:34 in reply to 1 [link] [source]

Save the file without the BOM.

Pragma encoding controls the internal text storage encoding and has nothing to do with anything external to the SQLite3 library code itself.

(3) By Gary (1codedebugger) on 2021-03-27 04:47:54 in reply to 2 [link] [source]

Thanks. It appears that cannot be done in Notepad unless using Windows 10. I'm not up to date. Guess a different editor is required.

(4) By Ryan Smith (cuz) on 2021-03-27 11:19:16 in reply to 3 [link] [source]

I didn't even know it can be done in Notepad in Windows 10!

Actually, I just checked, I still don't know how. Mind sharing me how to do so in Notepad in Win10?

(I know there are other options, but I love Notepad, it's light and minimalist, one step removed from a terminal, but also lacking in features. It's so very unlike the rest of Windows.)

(5) By TripeHound on 2021-03-27 14:49:22 in reply to 4 [link] [source]

In my version (Windows 10 Pro, 64 bit if it matters) the "Save As" dialog has an "Encoding" drop-down which includes "UTF-8" and "UTF-8 with BOM". I'm making a (perhaps unwarranted) assumption that the former does not include a BOM. I would try first with a new, empty file: it's conceivable that it won't remove a BOM that's already there, although I suspect it might.

(9) By Simon Slavin (slavin) on 2021-03-28 12:14:20 in reply to 5 [link] [source]

Notepad++ version 7.0 and above for Windows explicitly supports UTF-8 with and without BOM.

https://notepad-plus-plus.org/resources/

(8) By Gary (1codedebugger) on 2021-03-27 17:47:38 in reply to 4 [link] [source]

I was referring to what I read at the first link below; but I have subsequently checked on a Windows 10 Pro machine to which I have access and it is as was already stated by TripeHound.

The second link claims it can be changed by running a script but I did not try that out of fear I'd irreversibly mess something else up in the OS doing so.

https://www.bleepingcomputer.com/news/microsoft/windows-10-notepad-is-getting-better-utf-8-encoding-support/

https://www.tenforums.com/software-apps/110704-unreadable-non-ansi-characters-notepad.html

(10) By jose isaias cabrera (jicman) on 2021-03-29 13:09:56 in reply to 4 [link] [source]

(..., but I love Notepad, it's light and minimalist, one step removed from a terminal, but also lacking in features. It's so very unlike the rest of Windows.)

I don't care what anybody says, but that statement above is funny. :-)

(12) By Gary (1codedebugger) on 2021-09-17 05:51:25 in reply to 4 [link] [source]

I realize this is a bit old now but I think I understand what you mean about Notepad. I switched to Linux shortly after this post and use Kate text editor. I also use an older machine that runs Windows 7 and found out a couple days ago that Kate works on Windows. I remembered this exchange and thought it might be useful. I didn't get it from the MS store but their nightly builds page. If you turn off most of the features, it works much like Notepad but with a long undo/redo chain, tabs for each open file, and you can save all the open tabs as a session to be opened together next time.

(13) By anonymous on 2021-09-17 10:31:34 in reply to 12 [link] [source]

https://www.scintilla.org/SciTE.html
(The editor used for testing the Scintilla editor component)

Scintilla is also the basis for Notepad++
Multiple tabs
Sessions
Lightweight
Has a single executable version also
Portable
Supports many languages, syntax highlighting etc.
Very customizable
Fast
etc.

I have set it up to run the CLI SQLite.exe by selecting text and pressing a shortcut, results/output then goes to the output pane. Love it. I can post the SQLite/SciTE config here if others want to try it out.

Philip

(6) By Trudge on 2021-03-27 16:24:28 in reply to 3 [link] [source]

I program on a Mac and use BBEdit exclusively for programming. It is available for Windows. 13.5.4 Release Notes

(7) By Harald Hanche-Olsen (hanche) on 2021-03-27 17:40:59 in reply to 6 [source]

It is available for Windows.

Are you sure? It does not look that way to me.

But a bit of googling reveals this list of text editors, at least some of which run on Windows, and several of them free: 13 Best Text Editors to Speed up Your Workflow in 2021

(11) By anonymous on 2021-03-29 15:21:53 in reply to 7 [link] [source]

Yes, you're right. My bad. Not sure why I thought that.