SQLite Forum

bug report: table data display error when importing text file encoding utf-16le, utf-16be using .import command
Login

bug report: table data display error when importing text file encoding utf-16le, utf-16be using .import command

(1) By anonymous on 2020-09-12 11:19:46 [link] [source]

Hi, I downloaded sqlite 3.33.0 recently.

my operating system is Simplified Chinese WinXP + 203 hotfixes.

in sqlite 3.33.0, I inserted some Simplified Chinese Characters into table,
but Simplified Chinese Character displayed error.

do like below

C:>sqlite3.exe
sqlite> pragma encoding='utf16le';
sqlite> create table test (name, age integer);
sqlite> insert into test values ('tom', 10);
sqlite> insert into test values ('jerry', 20);
sqlite> insert into test values ('张三', 30);
sqlite> insert into test values ('李四', 40);
sqlite> select * from test;
tom|10
jerry|20
张三|30          ; Chinese displayed correctly.
李四|40          ; Chinese displayed correctly.

now, suppose text file utf16le.txt, encoding is utf-16le without/with BOM, its contents like:

tom|10
jerry|20
张三|30
李四|40

line delimiter is \n.


do like below:

C:>sqlite3.exe
sqlite> pragma encoding='utf16le';
sqlite> create table test2 (name, age integer);
sqlite> .import utf16le.txt test2       ;import utf16le.txt to test2.
sqlite> select * from test2;
t|       ;english displayed error, bug.
|        ;english displayed error, bug.
|        ;Chinese displayed error, bug.
|        ;Chinese displayed error, bug.

it has same bug that importing UTF-16be text file to UTF-16be table.

this bug found in sqlite 3.33.0 and earlier version.

(2) By Stephan Beal (stephan) on 2020-09-12 12:26:31 in reply to 1 [source]

@an admin user: this post can be deleted. The OP tried to edit it while it was pending moderation, resulting in a separate post being created and i didn't notice it until after approving this one.