[Fix][Tiered Storage] Eagerly Delete Offloaded Segments On Topic Deletion - #17915
Merged
Conversation
Contributor
1 task
dlg99
force-pushed
the
topic-deletion-take2
branch
from
October 4, 2022 21:26
67b921b to
ab7c5b3
Compare
…tion (apache#15914) * Truncate topic before deletion to avoid orphaned offloaded ledgers * CR feedback (cherry picked from commit 9026d19)
dlg99
force-pushed
the
topic-deletion-take2
branch
from
October 21, 2022 06:58
ab7c5b3 to
ba76543
Compare
Codecov Report
@@ Coverage Diff @@
## master #17915 +/- ##
=============================================
+ Coverage 34.91% 49.99% +15.07%
- Complexity 5707 7031 +1324
=============================================
Files 607 393 -214
Lines 53396 43489 -9907
Branches 5712 4470 -1242
=============================================
+ Hits 18644 21741 +3097
+ Misses 32119 19338 -12781
+ Partials 2633 2410 -223
Flags with carried forward coverage won't be shown. Click here to find out more.
|
dlg99
added a commit
to dlg99/pulsar
that referenced
this pull request
Oct 25, 2022
…tion (apache#17915) (cherry picked from commit 0854032)
dlg99
added a commit
to datastax/pulsar
that referenced
this pull request
Oct 31, 2022
…tion (apache#17915) (#152) (cherry picked from commit 0854032) Fixes apache#9962 ### Motivation Offloaded ledgers can be orphaned on topic deletion. This is a redo of apache#15914 which conflicted with concurrently merged apache#17736 thus resulting in apache#17889 . apache#17736 made a decision to not allow managed ledger trimming for the fenced mledgers because in many case fencing indicates a problems that should stop all operations on mledger. At the same time fencing is used before deletion starts, so trimming added to the deletion process cannot proceed. After discussion with @eolivelli I introduced new state, FencedForDeletion, which acts as Fenced state except for the trimming/deletion purposes. ### Modifications Topic to be truncated before deletion to delete offloaded ledgers properly and fail if truncation fails. ### Verifying this change local fork tests: dlg99#1 - [ ] Make sure that the change passes the CI checks. This change added integration tests ### Does this pull request potentially affect one of the following parts: *If `yes` was chosen, please highlight the changes* Nothing changed in the options but admin CLI will implicitly run truncate before topic delete. - Dependencies (does it add or upgrade a dependency): (yes / no) - The public API: (yes / no) - The schema: (yes / no / don't know) - The default values of configurations: (yes / no) - The wire protocol: (yes / no) - The rest endpoints: (yes / no) - The admin cli options: (yes / no) - Anything that affects deployment: (yes / no / don't know) ### Documentation Check the box below or label this PR directly. Need to update docs? - [ ] `doc-required` (Your PR needs to update docs and you will update later) - [x] `doc-not-needed` (Please explain why) - [ ] `doc` (Your PR contains doc changes) - [ ] `doc-complete` (Docs have been already added)
Member
|
@dlg99 @eolivelli This PR can cherry-pick into 2.9? Whether to introduce break change? |
Contributor
|
We cannot cherry pick this anywhere. |
5 tasks
zymap
added a commit
to zymap/pulsar
that referenced
this pull request
Dec 7, 2023
…xception --- ### Motivation apache#17915 changes the fix apache#17512 which lead the offload data is deleted when metadata store exception happened. Then the ledger can not be read. The logs shows ``` Failed to update offloaded metadata for the ledgerId 6197907, the offloaded data will not be cleaned up ``` But the ledger deleted. Then managed ledger failed to open it ``` Error opening ledger for reading at position 6197907:0 ```
zymap
added a commit
to zymap/pulsar
that referenced
this pull request
Dec 7, 2023
…xception --- ### Motivation apache#17915 changes the fix apache#17512 which lead the offload data is deleted when metadata store exception happened. Then the ledger can not be read. The logs shows ``` Failed to update offloaded metadata for the ledgerId 6197907, the offloaded data will not be cleaned up ``` But the ledger deleted. Then managed ledger failed to open it ``` Error opening ledger for reading at position 6197907:0 ```
15 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #9962
Motivation
Offloaded ledgers can be orphaned on topic deletion.
This is a redo of #15914 which conflicted with concurrently merged #17736 thus resulting in #17889 .
#17736 made a decision to not allow managed ledger trimming for the fenced mledgers because in many case fencing indicates a problems that should stop all operations on mledger. At the same time fencing is used before deletion starts, so trimming added to the deletion process cannot proceed.
After discussion with @eolivelli I introduced new state, FencedForDeletion, which acts as Fenced state except for the trimming/deletion purposes.
Modifications
Topic to be truncated before deletion to delete offloaded ledgers properly and fail if truncation fails.
Verifying this change
local fork tests: dlg99#1
This change added integration tests
Does this pull request potentially affect one of the following parts:
If
yeswas chosen, please highlight the changesNothing changed in the options but admin CLI will implicitly run truncate before topic delete.
Documentation
Check the box below or label this PR directly.
Need to update docs?
doc-required(Your PR needs to update docs and you will update later)
doc-not-needed(Please explain why)
doc(Your PR contains doc changes)
doc-complete(Docs have been already added)