SQLite Forum

bug report: table data display error when importing text file encoding utf-16le, utf-16be using .import command
Login
Hi, I downloaded sqlite 3.33.0 recently.

my operating system is Simplified Chinese WinXP + 203 hotfixes.<br>
<br>
in sqlite 3.33.0, I inserted some Simplified Chinese Characters into table,<br>
but Simplified Chinese Character displayed error.<br>

do like below

C:\>sqlite3.exe<br>
sqlite> pragma encoding='utf16le';<br>
sqlite> create table test (name, age integer);<br>
sqlite> insert into test values ('tom', 10);<br>
sqlite> insert into test values ('jerry', 20);<br>
sqlite> insert into test values ('张三', 30);<br>
sqlite> insert into test values ('李四', 40);<br>
sqlite> select * from test;<br>
tom|10<br>
jerry|20<br>
张三|30&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;; Chinese displayed correctly.<br>
李四|40&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;; Chinese displayed correctly.<br>

now, suppose text file utf16le.txt, encoding is utf-16le without/with BOM, its contents like:

tom&#124;10<br>
jerry&#124;20<br>
张三&#124;30<br>
李四&#124;40<br>

line delimiter is \\n.

<br>
do like below:

C:\>sqlite3.exe<br>
sqlite> pragma encoding='utf16le';<br>
sqlite> create table test2 (name, age integer);<br>
sqlite> .import utf16le.txt test2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;;import utf16le.txt to test2.<br>
sqlite> select * from test2;<br>
t|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;;english displayed error, bug.<br>
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;;english displayed error, bug.<br>
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;;Chinese displayed error, bug.<br>
|&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;;Chinese displayed error, bug.<br>
<br>
it has same bug that importing UTF-16be text file to UTF-16be table.<br>
<br>
this bug found in sqlite 3.33.0 and earlier version.<br>