SQLite Forum

Database Growing by itself to huge sizes
Login
I don't know if it's relevant to your problem, but there was an email-chain from 10 Jan 2020, titled "_Unexplained table bloat_" (which doesn't appear to be in the forum) with a possibly-similar sort of problem: the database size was much larger than the _apparent_ size of the content. That discussion drifted into many side topics, but the gist seems to be that what was being stored was essentially (large) strings/blocks-of-bytes with embedded `NUL` characters (`0x00` bytes) in them. The _apparent_ size only showed the length up to the "terminating NUL byte", but the database was storing the full collection of bytes.

I don't think the code that caused the problem in that case was shown, but the following (incomplete and untested) snippet should demonstrate a _possible_ cause:

```c
char bigBuffer[10000];
strcpy( bigBuffer, "A short string" );
sqlite3_bind_text( stmt, 1, bigBuffer, sizeof bigBuffer, NULL );
```

As (I think) I understand it, if the fourth parameter of [`sqlite3_bind_text`](https://www.sqlite.org/c3ref/bind_blob.html) has a positive value, then _that many bytes_ will be stored in the database (10,000 in this case), even if the block of memory given by the third parameter contains a zero-byte. From that page:

> If any NUL characters occurs at byte offsets less than the value of the fourth parameter then the resulting string value will contain embedded NULs. The result of expressions involving strings with embedded NULs is undefined.

Examining `select * ...` after the above would show only the string "A short string", but it would be consuming 10,000 bytes of space within the database. The SQL `length(...)` function would – I believe – only show 14, but the C API function [`sqlite3_column_bytes`](https://www.sqlite.org/c3ref/column_blob.html) should show 10,000 according to a reply Richard made in the aforementioned email chain:

> >
> > There's no way at all, to know the length of a text column with embedded
> > NULLs?
> >
>
> You can find the true length of a string in bytes from C-code using
the sqlite3_column_bytes() interface.  But I cannot, off-hand, think
of a way to do that from SQL.

It _might_ be worth reviewing your code to see if anything somewhat-similar might be occurring.