Motivation
- Sometimes due to overreplication, bookie contains ledgers which are not owned by that bookie anymore and that bookie is not part of the ensemble-list of those ledgers. In this case, GC finds out those overreplicated ledgers and
- deletes their index from dbStorage (rocksDB) and
- tries to delete them from entrylog files.
However, bookie doesn't delete them from entry-log files due to change made in #870 where bookie avoids deleting ledger if znode of that ledger exists.
Because of that bookie ends up storing large number entrylog files with ledgers which are owned by different bookies. It also cause OOM when GC tries to deal with large number of entry log files.
Fix
- OOM should be addressed by: 1949 or 1938
- And clean up overreplicated ledgers which are owned by different bookies should be fixed by this commit
I will create a PR with this fix.
Motivation
However, bookie doesn't delete them from entry-log files due to change made in #870 where bookie avoids deleting ledger if znode of that ledger exists.
Because of that bookie ends up storing large number entrylog files with ledgers which are owned by different bookies. It also cause OOM when GC tries to deal with large number of entry log files.
Fix
I will create a PR with this fix.