SQLite Forum

UTF-16 LE text delimited import

UTF-16 LE text delimited import

(1) By anonymous on 2020-05-19 13:51:05 [source]

Have a number of text delimited files downloaded in UTF-16 LE format, however when using the CLI and trying to .import I just get blank fields/??x (BOM) as the first record, and nothing else. Found a similar thread on Stack Overflow describing the exact problem


Have followed guides how to set the DB to UTF-16 mode, also read that apparently CSV can't be imported as UTF-16 but is there something I'm missing? Is there a way to import text delimited files but not as CSV regardless of .separator? If I convert the file to UTF-8 then it works fine, but would rather skip that manual conversion step in between.

Note using Windows 10 1903 with enhanced Notepad support but same problem on 1803 as well, happy to provide a sample file if anyone wants to try and recreate the issue.