PIP-45: Allow to configure metadata store URL in broker.conf - #13077
Merged
merlimat merged 11 commits intoDec 4, 2021
Conversation
merlimat
added
type/feature
merlimat
requested review from
codelipenghui,
eolivelli,
hangc0276,
jerrypeng,
lhotari and
rdhabalia
December 2, 2021 02:37
| # Configuration Store connection string | ||
| configurationStoreServers= | ||
| # The metadata store URL for the configuration data. If empty, we fall back to use metadataStoreUrl | ||
| configurationMetadataStoreUrl= |
Contributor
There was a problem hiding this comment.
It's will be a breaking change when we use the docker image? If we don't have this configuration in the broker.conf, we need to add PULSAR_PREFIX I think.
Contributor
Author
There was a problem hiding this comment.
I have moved all the deprecated configs in a section at the end of the file, to be less visible (and less confusing) but without breaking the compatibility.
codelipenghui
approved these changes
Dec 2, 2021
eolivelli
approved these changes
Dec 2, 2021
| import org.apache.pulsar.metadata.bookkeeper.BKCluster; | ||
|
|
||
|
|
||
| public class EmbeddedPulsarCluster implements AutoCloseable { |
Contributor
There was a problem hiding this comment.
Notes for the future:
It will be great to expose an official API for this.
It will help a lot downstream users.
PulsarStandalone Builder doesn't work well.
Contributor
Author
There was a problem hiding this comment.
Yes, I just started very simple as a way to use it as the base for unit tests, but with the idea of expanding it.
merlimat
force-pushed
the
pip-45-configure-metadata-store
branch
from
December 4, 2021 01:57
f6773e0 to
f7f3cde
Compare
Member
|
For the doc side, I've discussed with @codelipenghui, the following changes will be made after 2.9.0 is released:
Hi @merlimat any thoughts? |
fxbing
pushed a commit
to fxbing/pulsar
that referenced
this pull request
Dec 19, 2021
…13077) * PIP-45: Allow to configure metadata store URL in broker.conf * Removed unused imports * Register BK metadata-store scheme handler for BK client to accept config * Fixed default for loadBalancerOverrideBrokerNicSpeedGbps to Optional.empty() * Fixed registration of BK client driver handler * Fixed worker service bk client config * Fixed AdminTest * Fixed ledger id generator to handle badversion in graceful way * Fixed BookKeeperClientFactoryImplTest
1 task
This was referenced Jan 14, 2022
Contributor
|
@merlimat Should proxy also change zookeeperServers to metadataStoreUrl? |
This was referenced Jan 17, 2022
RobertIndie
added a commit
to RobertIndie/pulsar
that referenced
this pull request
Jan 17, 2022
Motivation As per apache#13077, `configurationStoreServers` need to be renamed to `configurationMetadataStoreUrl`. Modification * Add `configurationMetadataStoreUrl` setting to websocket * Deprecate `configurationStoreServers` in websocket Signed-off-by: Zike Yang <zkyang@streamnative.io>
1 task
RobertIndie
added a commit
to RobertIndie/pulsar
that referenced
this pull request
Jan 21, 2022
… cluster metadata Motivation As per apache#13077, `configurationStoreServers` need to be renamed to `configurationMetadataStoreUrl`. Modification * Add `configurationMetadataStore` to PulsarClusterMetadataSetup. Signed-off-by: Zike Yang <zkyang@streamnative.io>
Merged
1 task
Anonymitaet
added
doc-complete
codelipenghui
pushed a commit
that referenced
this pull request
Feb 7, 2022
Master Issue: #13077 ### Motivation Allow to configure the metadata store endpoint in proxy.conf. ### Modifications Deprecated few config options: zookeeperServers --> metadataStoreUrl configurationStoreServers --> configurationMetadataStoreUrl Moved all deprecated settings at the end of broker.conf.
Nicklee007
pushed a commit
to Nicklee007/pulsar
that referenced
this pull request
Apr 20, 2022
Master Issue: apache#13077 ### Motivation Allow to configure the metadata store endpoint in proxy.conf. ### Modifications Deprecated few config options: zookeeperServers --> metadataStoreUrl configurationStoreServers --> configurationMetadataStoreUrl Moved all deprecated settings at the end of broker.conf.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Allow to configure the metadata store endpoint in
broker.conf.Modifications
Deprecated few config options:
zookeeperServers-->metadataStoreUrlconfigurationStoreServers-->configurationMetadataStoreUrlMoved all deprecated settings at the end of
broker.conf.Added
EndToEndMetadataTestwhich spin up a cluster with brokers, bookies against all the supported metadata stores (ZK, local memory and RocksDB) and does basic produce/consume.