Skip to content

[fix][broker] Delete compacted ledger when topic is deleted - #21745

Merged
coderzc merged 10 commits into
apache:masterfrom
coderzc:fix_delete_compacted_ledger
Dec 29, 2023
Merged

[fix][broker] Delete compacted ledger when topic is deleted#21745
coderzc merged 10 commits into
apache:masterfrom
coderzc:fix_delete_compacted_ledger

Conversation

@coderzc

@coderzc coderzc commented Dec 18, 2023

Copy link
Copy Markdown
Member

Motivation

Currently, the compacted ledger doesn't delete when the topic is deleted, this will result in many orphaned ledgers, we should delete them.

Modifications

Delete compacted ledger when __compaction cursor is deleted.

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Dec 18, 2023
}
}

CompletableFuture<Void> cleanCompactedLedger() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have TopicCompactionService now, shall we add an new API to the TopicCompactionService?

@coderzc coderzc Dec 27, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is just to fix this old issue, it needs to be cherry-picked to the release branch, I think we can separately propose a PIP extending TopicCompactionService on the later version.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, sure.

@coderzc
coderzc force-pushed the fix_delete_compacted_ledger branch from ebe004a to c9d3068 Compare December 27, 2023 09:50

@BewareMyPower BewareMyPower left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition, to keep the same style, you can use assertEquals instead of Assert.assertEquals.

Comment thread pulsar-broker/src/test/java/org/apache/pulsar/compaction/CompactionTest.java Outdated
Comment thread pulsar-broker/src/test/java/org/apache/pulsar/compaction/CompactionTest.java Outdated
@coderzc
coderzc force-pushed the fix_delete_compacted_ledger branch from 66ecc41 to 5f96de3 Compare December 28, 2023 05:20
@coderzc
coderzc force-pushed the fix_delete_compacted_ledger branch from 5f96de3 to 3a2bf54 Compare December 28, 2023 05:21
@coderzc
coderzc merged commit 23391d3 into apache:master Dec 29, 2023
@coderzc
coderzc deleted the fix_delete_compacted_ledger branch December 29, 2023 06:39
coderzc added a commit to coderzc/pulsar that referenced this pull request Jan 4, 2024
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jan 4, 2024
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Jan 8, 2024
coderzc added a commit that referenced this pull request Jan 11, 2024
@coderzc coderzc modified the milestones: 3.3.0, 3.2.0 May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants