SQLite Forum

sqlite3_serialize cannot serialize an empty memory database
Login

sqlite3_serialize cannot serialize an empty memory database

(1) By TzankoMatev (tzanko) on 2024-01-20 11:21:56 [source]

An empty memory database has size 0. When you try to allocate a memory segment with 0 size, the function sqlite3_malloc64( sz ) returns NULL. As a result sqlite3_serialize returns NULL, which is interpreted as an error.

Current behaviour breaks probably many sqlite wrappers. Personally I checked the Go wrapper sqlite3-go and the Python sqlite3 library. In both cases they consider the NULL value to be an error and generate an error for the user.

I consider this behavior to be a bug. The correct behavior should be to return a valid representation of an empty database.

(2.1) By Stephan Beal (stephan) on 2024-01-20 14:02:50 edited from 2.0 in reply to 1 [link] [source]

The correct behavior should be to return a valid representation of an empty database.

An empty byte array is a valid representation of a db which has not undergone any writes, as can be demonstrated with:

$ touch foo.db; l foo.db; ./sqlite3 foo.db
-rw-rw-r-- 1 stephan stephan 0 Jan 20 14:51 foo.db
# this file is empty      ---^---
SQLite version 3.46.0 2024-01-19 16:51:34
Enter ".help" for usage hints.
sqlite> create table t(a);
sqlite> ^D

$ l foo.db
-rw-rw-r-- 1 stephan stephan 8192 Jan 20 14:51 foo.db

i.e. sqlite accepts that empty file as a database, rather than saying "db is corrupt."

The fact that serialize() returns NULL for an empty db is not an error, per se, but is ostensibly ambiguous. If it returns NULL you can use sqlite3_errcode() to determine whether the NULL is caused by an out-of-memory (in which case it returns SQLITE_NOMEM) or an empty db (in which case it returns 0).

(3) By Bo Lindbergh (_blgl_) on 2024-01-20 14:05:34 in reply to 1 [link] [source]

sqlite3_serialize returns the size even when memory allocation fails. This lets you distinguish a zero-byte result from an out-of-memory error.

(4) By TzankoMatev (tzanko) on 2024-01-20 15:03:06 in reply to 3 [link] [source]

That's good to know. But this behavior is unexpected and undocumented. According to the documentation returning NULL means that there was an error. Here is the only quote from the docs which I found that refers to a NULL return value:

A call to sqlite3_serialize(D,S,P,F) might return NULL even if the SQLITE_SERIALIZE_NOCOPY bit is omitted from argument F if a memory allocation error occurs.

Also the following part of the documentation is contradicted:

The sqlite3_serialize(D,S,P,F) interface returns a pointer to memory that is a serialization of the S database on database connection D. If P is not a NULL pointer, then the size of the database in bytes is written into *P.

For an ordinary on-disk database file, the serialization is just a copy of the disk file. For an in-memory database or a "TEMP" database, the serialization is the same sequence of bytes which would be written to disk if that database where backed up to disk.

If I save an empty database to disk using the CLI's .save command, I get a file with 4096 bytes. According to the documentation I should get the same byte sequence if I call sqlite3_serialize, and not a NULL pointer.

I looked at several sqlite3 wrappers for popular languages. So far I still haven't been able to find a wrapper which doesn't interpret the NULL return value as an error. Here are some examples:

sqlite3-go

CPython

sqlite-jdbc

(5) By Stephan Beal (stephan) on 2024-01-20 18:37:29 in reply to 4 [link] [source]

But this behavior is unexpected and undocumented.

A lack of documentation means that any given behavior is "not wrong."

In any case, Richard just modified serialize() to ensure that the db gets initialized to a non-0 size.

(6) By TzankoMatev (tzanko) on 2024-01-20 22:05:57 in reply to 5 [link] [source]

Wow, thanks to Richard for quick fix!