Skip to content

Multi-process reads for RocksDB among multiple Faust apps #330

Description

@wbarnha

I'd like to be able to setup multiple workers/Faust Apps that have the same name to read the same Faust Table entries. I may need to make some changes to python-rocksdb in order to achieve this.

I understand some of this functionality was added in #98, the note from Ask Solem in robinhood/faust#299 (comment) is no longer true.

Taken from https://github.com/aayushKumarJarvis/rocks-wiki/blob/master/RocksDB-FAQ.md:

Q: Can I write to RocksDB using multiple processes?

A: No. However, it can be opened in read-only mode from multiple processes.

Q: Does RocksDB support multi-process read access?

A: RocksDB support multi-process read only process without writing the database. This can be done by opening the database with DB::OpenForReadOnly() call

Could we have a Faust app configured to only read from a RocksDB file so that multiple processes could share one RocksDB file and only one process writes to the RocksDB file?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions