Skip to content

[improve][broker] Add parameter check for create/update cluster. - #19151

Merged
mattisonchao merged 8 commits into
masterfrom
paramter_check
Jan 9, 2023
Merged

[improve][broker] Add parameter check for create/update cluster.#19151
mattisonchao merged 8 commits into
masterfrom
paramter_check

Conversation

@mattisonchao

Copy link
Copy Markdown
Member

Motivation

In the current implementation, we didn't have any cluster data check for create/update cluster. it may cause some problems when the user sets the wrong URL.

Modifications

  • Add parameters check.

Verifying this change

  • Make sure that the change passes the CI checks.

Does this pull request potentially affect one of the following parts:

  • The REST endpoints

Documentation

  • doc-not-needed

@mattisonchao mattisonchao self-assigned this Jan 6, 2023
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Jan 6, 2023
@mattisonchao

Copy link
Copy Markdown
Member Author

@codelipenghui @eolivelli @Jason918 @congbobo184 @nicoloboschi
Would you guys mind taking a look?

@mattisonchao mattisonchao added this to the 2.12.0 milestone Jan 6, 2023
@mattisonchao mattisonchao reopened this Jan 6, 2023
@codecov-commenter

codecov-commenter commented Jan 6, 2023

Copy link
Copy Markdown

Codecov Report

Merging #19151 (3e63d54) into master (492a9c3) will decrease coverage by 3.16%.
The diff coverage is 47.82%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #19151      +/-   ##
============================================
- Coverage     47.46%   44.29%   -3.17%     
+ Complexity    10727    10057     -670     
============================================
  Files           711      713       +2     
  Lines         69456    69683     +227     
  Branches       7452     7483      +31     
============================================
- Hits          32964    30865    -2099     
- Misses        32810    35206    +2396     
+ Partials       3682     3612      -70     
Flag Coverage Δ
unittests 44.29% <47.82%> (-3.17%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...va/org/apache/pulsar/client/impl/ConsumerImpl.java 15.06% <0.00%> (-0.04%) ⬇️
.../apache/pulsar/broker/admin/impl/ClustersBase.java 76.68% <81.81%> (+0.02%) ⬆️
...lsar/client/impl/conf/ClientConfigurationData.java 96.66% <100.00%> (ø)
...pache/pulsar/client/impl/CompactionReaderImpl.java 0.00% <0.00%> (-90.00%) ⬇️
...g/apache/pulsar/client/impl/RawBatchConverter.java 3.12% <0.00%> (-89.07%) ⬇️
.../pulsar/compaction/StrategicTwoPhaseCompactor.java 0.00% <0.00%> (-76.20%) ⬇️
...sar/broker/stats/metrics/ManagedLedgerMetrics.java 23.88% <0.00%> (-76.12%) ⬇️
...lsar/client/impl/RawBatchMessageContainerImpl.java 0.00% <0.00%> (-73.14%) ⬇️
...ice/streamingdispatch/PendingReadEntryRequest.java 0.00% <0.00%> (-68.19%) ⬇️
.../pulsar/broker/service/SharedConsumerAssignor.java 3.70% <0.00%> (-64.82%) ⬇️
... and 102 more

@mattisonchao mattisonchao added release/important-notice The changes which are important should be mentioned in the release note and removed release/important-notice The changes which are important should be mentioned in the release note labels Jan 8, 2023
.allowedClusters(Collections.singleton(cluster))
.build();
ClusterDataImpl clusterData = ClusterDataImpl.builder().serviceUrl(cluster).build();
ClusterDataImpl clusterData = ClusterDataImpl.builder().serviceUrl("http://example.pulsar").build();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a breaking change? I haven't reviewed the details.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not a breaking change; we just didn't have data validation.

@Technoboy- Technoboy- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lhotari

lhotari commented Jan 12, 2023

Copy link
Copy Markdown
Member

@mattisonchao Is this PR a potential breaking change? Please check #19212 .
("Pulsar CI Flaky" jobs have been failing for several days with OOME after this PR was merged: https://github.com/apache/pulsar/actions/workflows/pulsar-ci-flaky.yaml . The OOME was a side-effect of the test class that broke because of the validation change.)

@lhotari

lhotari commented Jan 12, 2023

Copy link
Copy Markdown
Member

@mattisonchao I wonder if a "https" url would need to be accepted in the serviceUrl just because of backwards compatibility reasons?

Comment on lines +410 to +428
public void checkPropertiesIfPresent() throws IllegalArgumentException {
URIPreconditions.checkURIIfPresent(getServiceUrl(),
uri -> Objects.equals(uri.getScheme(), "http"),
"Illegal service url, example: http://pulsar.example.com:8080");
URIPreconditions.checkURIIfPresent(getServiceUrlTls(),
uri -> Objects.equals(uri.getScheme(), "https"),
"Illegal service tls url, example: https://pulsar.example.com:8443");
URIPreconditions.checkURIIfPresent(getBrokerServiceUrl(),
uri -> Objects.equals(uri.getScheme(), "pulsar"),
"Illegal broker service url, example: pulsar://pulsar.example.com:6650");
URIPreconditions.checkURIIfPresent(getBrokerServiceUrlTls(),
uri -> Objects.equals(uri.getScheme(), "pulsar+ssl"),
"Illegal broker service tls url, example: pulsar+ssl://pulsar.example.com:6651");
URIPreconditions.checkURIIfPresent(getProxyServiceUrl(),
uri -> Objects.equals(uri.getScheme(), "pulsar")
|| Objects.equals(uri.getScheme(), "pulsar+ssl"),
"Illegal proxy service url, example: pulsar+ssl://ats-proxy.example.com:4443 "
+ "or pulsar://ats-proxy.example.com:4080");
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we document these requirements? Also, what would happen if a user had already created a cluster in a bad state?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/broker doc-not-needed Your PR changes do not impact docs ready-to-test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants