Is your enhancement request related to a problem? Please describe.
Currently task internalTrimConsumedLedgers will delete all closed ledgers which have reached retention except current ledger.
If the current ledger does not closed, the ledger will not deleted even retention reached.
Current ledger will be close in the follow condition:
- entries in the ledger reaches
managedLedgerMaxEntriesPerLedger (default is 50000).
- total size of the ledger reached
managedLedgerMaxSizePerLedgerMbytes (default is 2048MB)
managedLedgerMaxLedgerRolloverTimeMinutes (default is 4 hours)
That means each partitioned topic could retain 2GB storage size at most in current ledger for 4 hours, which could not be deleted after retention reached due to not closed. And in some situation like using shared bookkeeper, we hope the disk storage could be reclaimed more quickly, but retention time is not working. (related issue: #6935 #6639)
Describe the solution you'd like
Lower managedLedgerMaxLedgerRolloverTimeMinutes could be a solution, but this will roll over all current ledgers even retention is not reached.
Could we close current ledger when it reach retention? Add a judgement in internalTrimConsumedLedgers, if current ledger reach retention time or size, close current ledger and create new ledger, so that the closed ledger could be deleted in the next trim task. Of coure managedLedgerMinLedgerRolloverTimeMinutes could take effect to avoid close ledger too frequently.
Is your enhancement request related to a problem? Please describe.
Currently task
internalTrimConsumedLedgerswill delete all closed ledgers which have reached retention except current ledger.If the current ledger does not closed, the ledger will not deleted even retention reached.
Current ledger will be close in the follow condition:
managedLedgerMaxEntriesPerLedger(default is 50000).managedLedgerMaxSizePerLedgerMbytes(default is 2048MB)managedLedgerMaxLedgerRolloverTimeMinutes(default is 4 hours)That means each partitioned topic could retain 2GB storage size at most in current ledger for 4 hours, which could not be deleted after retention reached due to not closed. And in some situation like using shared bookkeeper, we hope the disk storage could be reclaimed more quickly, but retention time is not working. (related issue: #6935 #6639)
Describe the solution you'd like
Lower
managedLedgerMaxLedgerRolloverTimeMinutescould be a solution, but this will roll over all current ledgers even retention is not reached.Could we close current ledger when it reach retention? Add a judgement in
internalTrimConsumedLedgers, if current ledger reach retention time or size, close current ledger and create new ledger, so that the closed ledger could be deleted in the next trim task. Of couremanagedLedgerMinLedgerRolloverTimeMinutescould take effect to avoid close ledger too frequently.