added Litestream Replication

[Alex-CodeLab]

Litestream.io enables easy SQLite replication for continuous backups

[Alex-CodeLab]

Litestream requires periodic but short write locks on the database in order to sync changes. SQLite will return an error by default if your application tries to obtain a write lock at the same time. To prevent this, your application will need to set the busy_timeout pragma when it initializes the database connection.

How does c-lightning handle write-locks now?

[rustyrussell]

I think this will break c-lightning: we do not expect db operations to fail at all. We would definitely want to block in this case.

[Alex-CodeLab]

I think this will break c-lightning: we do not expect db operations to fail at all. We would definitely want to block in this case.

I noticed the same issue was discussed here already #4207 (comment)
Setting PRAGMA busy_timeout is a reasonable solution.

-- edit --
seems the quote above about periodic locks was outdated

Originally, Litestream used the write locks to demarcate the transactions in the WAL but that was fixed in v0.3.4.
Now, the only time Litestream requires a write lock is during checkpointing which occurs after ~40MB of page writes.

[rustyrussell]

OK, I (trivially) rebased on master, and added commits so that we set the busy timeout. I also remove the copy operation in our tests, which should actually exercise this a little.

[rustyrussell]

Trivial rebase.

[rustyrussell]

YA trivial rebase.

[rustyrussell]

Ack 858156a