Replace redis dependency?

[spacebear21]

Redis moved to closed-source and the rust crate is being disputed. redis-rs/redis-rs#1419

[DanGould]

valkey is the obvious candidate that comes to mind.

[arminsabouri]

I would put my vote in for sled. Its simple to work with and fast.

[DanGould]

Unfortunately sled hasn't been maintained for 2 years which I reasoned was OK for payjoin-cli since it's a bit of a demo tool, but probably not ok for the aspirationally production-hardened payjoin-directory.

Redb is an alternative I hear come up a lot for sled, but no modern version of it complies with our MSRV. Now, I'd be ok bumping MSRV just for payjoin-directory but NOT for the payjoin crate. Not sure how much trouble multiple crates with different MSRV would be for our repo.

[DanGould]

Noting this isn't really so much tech debt as it is maintenance. When the dependency was taken on it was state of the art, and the state of the art changed.

Not really tech debt to pay back, but regular maintenance

[DanGould]

On this topic, Does it then make sense to use sqlite's in-memory :memory: database to keep our dependency tree simpler if we're going to use it for payjoin-cli #867 #873? That seems easier than writing and maintaining an in-memory database of our own, and devs can have an easier time working across the repo with some cross-over in understanding of persistence that depends on the same system between crates.

[nothingmuch]

an in memory database is just a hashmap... the tricky part is having listeners etc which sqlite can't really help with

i've started working this last week, still kind of in refactoring hell:

1. DirectoryService struct with an ohttp::Server and a generic Db trait, which is based on the DbPool public API with some changes
2. feature gating of redis in directory
3. a waitmap like structure with future::shared similar to how ohttp-relay works (but a bit simpler in some respects)

not yet done:

abstracting the in memory waitmap to have a generic KV store which is just HashMap<ShortId, Payload> (where struct Payload([u8; 7168])) for the in memory DB. this can be made persistent using files on disk, rocksdb with expiry, sqlite, redis, ... see #775 for more bikeshedding of rhis part