Skip to content

[PIP-82] [pulsar-broker] incorporate review feedback - #10201

Merged
jerrypeng merged 1 commit into
apache:masterfrom
bharanic-dev:rq-transport-misc
Apr 17, 2021
Merged

[PIP-82] [pulsar-broker] incorporate review feedback#10201
jerrypeng merged 1 commit into
apache:masterfrom
bharanic-dev:rq-transport-misc

Conversation

@bharanic-dev

Copy link
Copy Markdown
Contributor

Motivation

Follow up and incorporate the review feedback.

Modifications

  • Have lightproto generate sources to a different directory so it doesn't come in the way of protobuf generated sources.
  • make resourceUsageTransportPublishTopicName internal(remove from config)
  • drop old/stale resource-usage messages
  • fix a race condition in creating tenant/namespace for resource-usage when multiple brokers attempt to do it.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.

)
private String resourceUsageTransportClassName = "";

@FieldContext(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the rationale for making this not configurable?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jerrypeng while reviewing the previous PR(#10008), @codelipenghui and @315157973 gave feedback that the topic name is internal implementation detail that the user doesn't need to be exposed to. I did not have a strong reason to go the other way, so made this change. We can always make it configurable in the future, if we find that there is a use-case for it.

private final Map<String, ResourceUsageConsumer>
consumerMap = new ConcurrentHashMap<String, ResourceUsageConsumer>();

private long staleMessageCount = 0;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This variable doesn't seem to be used anywhere?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the variable with the intention of adding a unit test for it. But could not figure out a easy way to do so. I am open to ideas if you have suggestions on how I can add one. Else, I will remove this variable.

@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

1 similar comment
@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@sijie sijie added this to the 2.8.0 milestone Apr 13, 2021
@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

5 similar comments
@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

- Have lightproto generate sources to a different directory so it doesn't come in the way of protobuf generated sources.
- make resourceUsageTransportPublishTopicName internal(remove from config)
- drop old/stale resource-usage messages
- fix a race condition in creating tenant/namespace for resource-usage when multiple brokers attempt to do it.
@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

2 similar comments
@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev

Copy link
Copy Markdown
Contributor Author

/pulsarbot run-failure-checks

@jerrypeng
jerrypeng merged commit 1cc782f into apache:master Apr 17, 2021
@linlinnn

Copy link
Copy Markdown
Contributor

@jerrypeng CICD seems broken for this merge

@linlinnn

Copy link
Copy Markdown
Contributor

image

@linlinnn

linlinnn commented Apr 17, 2021

Copy link
Copy Markdown
Contributor

Seems should remove this line
this.conf.setResourceUsageTransportPublishTopicName(INTERNAL_TOPIC);
but actually remove this.conf.setResourceUsageTransportClassName("org.apache.pulsar.broker.resourcegroup.ResourceUsageTransportManager");

Same in ResourceGroupUsageAggregationTest

linlinnn added a commit to linlinnn/pulsar that referenced this pull request Apr 17, 2021
@linlinnn

Copy link
Copy Markdown
Contributor

@merlimat PTAL

lhotari added a commit to lhotari/pulsar that referenced this pull request Apr 17, 2021
@lhotari

lhotari commented Apr 17, 2021

Copy link
Copy Markdown
Member

@linlinnn @jerrypeng @codelipenghui @eolivelli @merlimat @aahmed-se I created #10250 to fix the compilation issues in master branch.

aahmed-se pushed a commit that referenced this pull request Apr 17, 2021
if (!tenantList.contains(tenant)) {
admin.tenants().createTenant(tenant,
new TenantInfo(Sets.newHashSet(config.getSuperUserRoles()), Sets.newHashSet(cluster)));
try {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To add try/catch is a good solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants