When sqlite3_backup_remaining() returns 0...
(1) By Michael Allman (msa) on 2021-12-08 20:08:13 [link] [source]
It seems clear that if sqlite3_backup_step() returns SQLITE_DONE and no further changes are made to the source database in the interim, then sqlite3_backup_remaining() will return 0.
If sqlite3_backup_remaining() returns 0 and no changes were made to the source database since the last call to sqlite3_backup_step(), does that mean that the last call to sqlite3_backup_step() returned SQLITE_DONE?
(2) By Larry Brasfield (larrybr) on 2021-12-08 20:57:02 in reply to 1 [link] [source]
If sqlite3_backup_remaining() returns 0 and no changes were made to the source database since the last call to sqlite3_backup_step(), does that mean that the last call to sqlite3_backup_step() returned SQLITE_DONE?
Michael, this question is a bit troubling. It implies that you may want to alter your use of the Backup API depending on what other DB clients may do. (Either that, or it springs from idle curiosity.) What is documented and safe, with guaranteed results (including absent results), is that the Backup API set can be used with the documented call sequence. Whether you complete the backup or not, sqlite3_backup_finish() should be called unless you mean to leak memory. The backup is not complete, and its transaction should not be assumed to be committed (as opposed to rolled back) until sqlite3_backup_step() says it is done. These facts are independent of activity by other DB clients.
The 0 return from sqlite3_backup_remaining() may or may not mean that the backup is, in fact, complete and committed. That is an implementation detail, one which could change without affecting the documented API (which is the supported API with guaranteed effects.)
While I doubt that this implementation detail would be affected by any activity on other connections (outside of the source transaction), I do not think any meaning or utility should be attached to this behavior subtlety. I can also represent that unnecessary guarantees of implementation detail are generally avoided by the SQLite project management.
(3) By Michael Allman (msa) on 2021-12-09 00:51:51 in reply to 2 [link] [source]
Hi Larry.
Let me clarify that I have no desire to use the database backup API in a manner other than intended. My question comes from finding the best way to communicate completion to a client callback that also receives progress from sqlite3_backup_remaining().
In a scenario like Example 2 from Using the SQLite Backup API, if a client has provided a callback to report backup progress that looks like
void (*xProgress)(int, int)
and it receives a call where remaining == 0, can it infer that the last call to sqlite3_backup_step() returned SQLITE_DONE?
As an alternative, we could use a callback like
void (*xProgress)(int, int, bool)
where the last parameter is set to true if sqlite3_backup_step() returns SQLITE_DONE. Otherwise, it's false. The client uses that "done" parameter to signal completion, not remaining == 0.
Note, we understand that "done" does not mean "committed". It just means sqlite3_backup_step() returned SQLITE_DONE.
What do you think?
(4) By Larry Brasfield (larrybr) on 2021-12-09 01:49:37 in reply to 3 [source]
I think you can reasonably infer, from the sqlite3_backup_finish() doc language, that once sqlite3_backup_step() has returned SQLITE_DONE the backup is complete and committed, leaving only resource recovery to done by sqlite3_backup_finish().
What it not at all clear, or even vaguely inferable, from the docs is whether a 0 return from sqlite3_backup_remaining() means that the commit has occurred. It certainly means that no more ordinary copying remains. But whether other work remains to complete the transaction on the destination DB (file), prior to sqlite3_backup_step() returning SQLITE_DONE, is not to be found in the docs.
Looking behind the interface, (which I disrecommend except to satisfy idle curiosity), I see the whole backup API as a way to break up what may be lengthy file I/O operations into smaller chunks so that a library client need not suffer lengthy delay from call to return. I would not be at all surprised to find: that activity associated with committing a big (by transferred byte count) transaction also takes awhile, and that this might be deferred until the last sqlite3_backup_step() call, (the one that will return SQLITE_DONE), occurs; and that the 0 return from sqlite3_backup_remaining() could occur before that last step. Or, it would be unsurprising to find that all the work, both the last few page copies and the destination commit I/O, gets done during that last step.
If you can see in the docs some way to rule out one of those unsurprising alternatives, I would be interested in seeing it. (If that is behavior to be relied upon, it should be clarified.) If you have the same difficulty as me distinguishing which alternative corresponds to the implementation, then I suggest you use sqlite3_backup_remaining() returns only as an estimate of how much time-consuming work remains to be done.