SQLite Forum

SQLite Binary Log support!
Login
We have session/changeset/patchset for sqlite, but they are quit limited compare to Mysql Binary Log.


If there is a function generate a binary log for each write transition, then late able to replay the log on a slave instance and get same result, then this can be done with it:


a) save binary log into kafka or other message queue, then you get a quit scalable master/slave cluster.


b) Use Raft leader replicate binary log instead WAL frame,  the raft entity size will be much small compare to https://github.com/canonical/dqlite.


c) You can setup a memory database with binary log store into kafka, use it with Raft you get a high performance and high availability cluster.


Here is a simple planning to add binary log:


1) when start a write transaction create a binary log entity object
2) for each sqlite_step call from user code, save the SQL and bind value into  binary log entity object. (check duplicate SQL and value, save as ref)
3) when commit write transaction, save serialized binary log entity object into memory block. 
4) maybe add changed page hash into the log object, to prevent misuse. (and a hash chain in order for each binary log entity)


To replay it to a database handle, first compare hash, then exec sqlite3_step in order.  Before each step do the SQL statement init if not yet, and bind value, compare changed page hash.


Is this plan feasible?