SQLite Forum

Freeing memory used in sqlite£-result_blob
Login

Freeing memory used in sqlite3_result_blob

(1.2) By curmudgeon on 2020-07-27 09:08:29 edited from 1.1 [link] [source]

Suppose I write a user defined function that basically just declares and populates a blob e.g.

char *blob=(char*)malloc(50);
// populate blob

should I end the function

sqlite3_result_blob(ctx,blob,50,SQLITE_TRANSIENT);
free(blob);

or

sqlite3_result_blob(ctx,blob,50,free);

While the latter would save sqlite having to create a copy of the blob I'm a bit worried about the function being called multiple times in the same sql statement e.g.

select fct(..), fct(..), fct(..), fct(..), fct(..);

and the frees building up. Am I worrying about nothing?

Also, is the use of SQLITE_TRANSIENT a necessity if the memory will be automatically freed at the end of the function? e.g. in the case of

char blob[50]={...};

(2.2) By Dan Kennedy (dan) on 2020-07-27 10:55:52 edited from 2.1 in reply to 1.2 [link] [source]

should I end the function

sqlite3_result_blob(ctx,blob,50,SQLITE_TRANSIENT); free(blob);

or

sqlite3_result_blob(ctx,blob,50,free);

While the latter would save sqlite having to create a copy of the blob I'm a bit worried about the function being called multiple times in the same sql statement e.g.

select fct(..), fct(..), fct(..), fct(..), fct(..);

and the frees building up. Am I worrying about nothing?

Either of those two is correct. The difference in performance is most likely too small to worry about, unless your application is super performance-critical. No need to worry about the "frees building up" either. Using SQLITE_TRANSIENT might make the code easier to grok, but I think it's a matter of taste more than anything.

Also, is the use of SQLITE_TRANSIENT a necessity if the memory will be automatically freed at the end of the function? e.g. in the case of

char blob[50]={...};

It is, yes. Otherwise SQLite will try to use the buffer after it has been automatically freed.

(3) By curmudgeon on 2020-07-27 11:08:32 in reply to 2.2 [link] [source]

Thanks Dan.

(4) By Gunter Hick (gunter_hick) on 2020-07-27 13:34:46 in reply to 1.2 [source]

If a referenced memory object never goes away and keeps its value, use SQLITE_STATIC.

e.g.: static const char id[] = "1.2.3.4.5";

If a referenced memory object goes away or looses its value, use SQLITE_TRANSIENT.

e.g. examples 1 (malloc & free in user function) and 3 (stack variable) in the original post

Otherwise, pass a destructor function.

e.g. example 2 (malloc, pass destructor) in the original post

(5) By Gunter Hick (gunter_hick) on 2020-07-27 13:49:41 in reply to 1.2 [link] [source]

The most memory efficient and fastest way would be example 2.

Assuming you are calling the user defined function twice, the memory subsystem calling sequence would be:

malloc #1 (by fct)
malloc #1 (by SQLite)
memcpy #1 (by SQLite)
free   #1 (by fct)
malloc #2 (by fct - possibly the same address as before)
malloc #2 (by SQLite)
memcpy #2 (by SQLite)
free   #2 (by fct)
...
free   #1 (by SQLite)
free   #2 (by SQLite)

total 4x malloc, 4x free, 2x memcpy, max 3 blobs concurrently allocated

versus

malloc #1 (by fct)
malloc #2 (by fct)
...
free   #1 (by SQLite)
free   #2 (by SQLite)

total 2x malloc, 2x free, max 2 blobs concurrently allocated

versus

(allocate on stack in fct)
malloc #1 (by SQLite)
memcpy #1 (by SQLite)
(allocate on stack in fct)
malloc #2 (by SQLite)
memcpy #2 (by SQLite)
...
free   #1 (by SQLite)
free   #2 (by SQLite)

total 2x malloc, 2x memcpy, max 2 blobs concurrently allocated

(6) By curmudgeon on 2020-07-27 16:27:38 in reply to 5 [link] [source]

Thanks Gunter.

(7) By Keith Medcalf (kmedcalf) on 2020-07-27 16:42:53 in reply to 5 [link] [source]

Also do not forget the significance that a memory block allocated by SQLite3 via the SQLITE_TRANSIENT method is a memory block that "belongs" to SQLite3 and comes from SQLite3's heap, and that SQLite3 can realloc (for example) since it knows all about how such memory blocks are maintained and may be maipulated.

When you pass a pointer and a destructor, all SQLite3 has is a "foreign" pointer and a destructor function. The only thing that SQLite3 can do with the "foreign" memory is to read / write (within the bounds it has been told) / call the destructor.

There may exist cases where this difference is significant. This applies equally to "text" since there is no physical difference between a blob and text, merely a symantic (touchy feely) difference. You will note that all other data types (integer/real) are passed by coping to a thing under SQLite3's control in the same manner as SQLITE_TRANSIENT (just that the size of the memory area is much smaller (8 bytes or less) and different "feelies" are attached to it).

(8) By Keith Medcalf (kmedcalf) on 2020-07-27 16:45:20 in reply to 5 [link] [source]

Also do not forget the significance that a memory block allocated by SQLite3 via the SQLITE_TRANSIENT method is a memory block that "belongs" to SQLite3 and comes from SQLite3's heap, and that SQLite3 can realloc (for example) since it knows all about how such memory blocks are maintained and may be maipulated.

When you pass a pointer and a destructor, all SQLite3 has is a "foreign" pointer and a destructor function. The only thing that SQLite3 can do with the "foreign" memory is to read / write (within the bounds it has been told) / call the destructor.

There may exist cases where this difference is significant. This applies equally to "text" since there is no physical difference between a blob and text, merely a symantic (touchy feely) difference. You will note that all other data types (integer/real) are passed by coping to a thing under SQLite3's control in the same manner as SQLITE_TRANSIENT (just that the size of the memory area is much smaller (8 bytes or less) and different "feelies" are attached to it).

(9) By curmudgeon on 2020-07-28 06:45:22 in reply to 8 [link] [source]

Thank you Keith.

(10) By curmudgeon on 2020-07-28 13:07:41 in reply to 8 [link] [source]

Should it be
sqlite3_result_blob(...,free);
or
sqlite3_result_blob(...,&free);

(11) By Larry Brasfield (LarryBrasfield) on 2020-07-28 13:25:46 in reply to 10 [link] [source]

Oddly, in the C language, those two forms have the same meaning.

(12) By curmudgeon on 2020-07-28 15:45:13 in reply to 11 [link] [source]

I wondered why they both worked.