[improve][misc] Replace dependencies on PositionImpl with Position interface - #22891
Merged
Conversation
lhotari
requested review from
BewareMyPower,
codelipenghui,
hangc0276,
horizonzy,
merlimat and
zymap
June 11, 2024 15:53
… casts everywhere
lhotari
force-pushed
the
lh-positionimpl-to-position
branch
from
June 11, 2024 16:03
a6baa92 to
5586660
Compare
lhotari
added
type/cleanup
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #22891 +/- ##
============================================
- Coverage 73.57% 73.27% -0.31%
- Complexity 32624 33028 +404
============================================
Files 1877 1896 +19
Lines 139502 142008 +2506
Branches 15299 15565 +266
============================================
+ Hits 102638 104050 +1412
- Misses 28908 29942 +1034
- Partials 7956 8016 +60
Flags with carried forward coverage won't be shown. Click here to find out more.
|
Member
|
This change is so massive that I think docs are in fact required to explain all of the changes in a TL/DR manner. |
merlimat
approved these changes
Jun 12, 2024
Member
Author
@dave2wave The reason why this change is massive is mainly due to the PositionImpl -> Position/PositionFactory changes. The PR description already contains the explanation in a TL/DR manner in "Modifications". |
Co-authored-by: Matteo Merli <mmerli@apache.org>
4 tasks
hanmz
pushed a commit
to hanmz/pulsar
that referenced
this pull request
Feb 12, 2025
…terface (apache#22891) Co-authored-by: Matteo Merli <mmerli@apache.org>
Demogorgon314
pushed a commit
to Demogorgon314/kop
that referenced
this pull request
Apr 15, 2026
…osition implementation for compaction (streamnative#606) ### Motivation apache/pulsar#22891 introduces some changes for the PositionImpl. Now we need to use the Position interface instead of the PositionImpl. ### Modifications - Refactor to use Position instead of PositionImpl - apache/pulsar#22572 uses `getLastDispatchablePosition` instead of `getMaxPosition` to return the last message id, this patch implements the new method for `KopPersistentTopic`. - apache/pulsar#22838 changed the result for `checkTopicExists`. This PR also adapts to changes. - apache/pulsar#22882 removed the method `org.apache.pulsar.broker.service.Producer.updateRates(int numOfMessages, long msgSizeInBytes)`. We need to use `producer.getStats().recordMsgIn` instead. --------- Co-authored-by: Yunze Xu <xyzinfernity@163.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The Pulsar Managed Ledger module contains an interface
org.apache.bookkeeper.mledger.Position.The general recommendation of using interfaces is to depend on the interface and not on implementation details.
However, this principle isn't followed in the code and there are a lot of casting to
PositionImplin the code and tight coupling to the implementation.Another problem with the current code base is that batch index acknowledgement state ("ack set") is also coupled to the Position implementation. This PR provides a solution where "ack set" handling is decoupled from the
Positioninterface without changing the architecture. It will continue to be possible to create a Position instance that carries and holds the "ack set" state for those execution paths that depend on this.The motivation of this PR is to cleanup code before Pulsar 4.0 which is scheduled for October. The current master branch will most likely become 4.0. Since 4.0 will be the next LTS release, we would like to complete larger changes in the code base before the release.
The Position interface isn't exposed in Pulsar end user applications. Therefore this change can be considered as internal cleanup and refactoring which doesn't necessary require a PIP decision before proceeding. I'll bring this PR up to discussion on the mailing list regardless so this change doesn't come as a surprise to other contributors.
Third party Pulsar broker plugins depending on managed ledger interfaces will need to be adapted. However, this is a simple process. It's a simple replacement of
PositionImplwithPositionand usingPositionFactoryfor the creation of Position instances.After this PR, I'm planning to continue cleaning Managed Ledger the
ManagedLedgerandManagedCursorinterfaces. This PR is a pre-requisite for that work. Decoupling Pulsar core from Managed Ledger implementations will open up ways to add new Managed Ledger implementations in the future and make it pluggable.Modifications
PositionImplwithPositionand removePositionImplcompletely to remove any confusion in the future.PositionFactoryfor creating new instances so that there's no need for coupling toPositionImplPositionwith a solution (AckSetState) that doesn't change the current architecture.Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: lhotari#193