Skip to content

[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRemoveBundleAndLoad - #17487

Merged
Technoboy- merged 4 commits into
apache:masterfrom
poorbarcode:flaky/testModularLoadManagerRemoveBundleAndLoad_
Sep 8, 2022
Merged

[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRemoveBundleAndLoad#17487
Technoboy- merged 4 commits into
apache:masterfrom
poorbarcode:flaky/testModularLoadManagerRemoveBundleAndLoad_

Conversation

@poorbarcode

@poorbarcode poorbarcode commented Sep 6, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Fixes #17386

Motivation

In this test, we expect it works like this:

  1. persistent broker data to ZK
  2. receive notice: broker data changes, and refresh bundle data in memory
  3. persistent bundle data which already updated in memory to ZK
  4. verify bundle data persist correctly
  5. delete namespace
  6. verify bundle data deleted

But the flow works like this:

  1. persistent broker data to ZK
  2. (High light) race condition
    2-x. receive notice: broker data changes, and refresh bundle data in memory
    2-x. persistent bundle data to ZK ( If the data in memory is not updated, it will not be written to ZK )
  3. ......

Modifications

  • guarantee step-3 is always executed after step-2
  • In this test, the original codes mock ModularLoadManagerWrapper to make isCentralized always return true. But this mock is completely unnecessary because that's ModularLoadManagerWrapper.isCentralized was exactly always returned true, so remove the mock.

Documentation

  • doc-required

  • doc-not-needed

  • doc

  • doc-complete

@poorbarcode

poorbarcode commented Sep 6, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

This PR should merge into these branches (because this test was appended in branch-2.11):

  • branch-2.11
  • master

@codelipenghui codelipenghui added this to the 2.12.0 milestone Sep 6, 2022
@codelipenghui

codelipenghui commented Sep 6, 2022

Copy link
Copy Markdown
Contributor

@heesung-sn Please help review this PR, thanks.

gaoran10
gaoran10 reviewed Sep 6, 2022
View reviewed changes
Comment thread pulsar-broker/src/test/java/org/apache/pulsar/broker/namespace/NamespaceServiceTest.java Outdated
}

private void waitResourceDataUpdateToZK(
LoadManager loadManager, Predicate<LoadData> utilChecker) throws Exception {

@gaoran10 gaoran10 Sep 6, 2022

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems the param utilChecker is useless.

@poorbarcode poorbarcode Sep 6, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was left over from the previous solution and I forgot to delete it. Already deleted, Thanks

gaoran10
gaoran10 approved these changes Sep 6, 2022
View reviewed changes

@gaoran10 gaoran10 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! LGTM

Demogorgon314
Demogorgon314 approved these changes Sep 6, 2022
View reviewed changes

@Demogorgon314 Demogorgon314 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@github-actions

github-actions Bot commented Sep 6, 2022

Copy link
Copy Markdown

@poorbarcode Please provide a correct documentation label for your PR.
Instructions see Pulsar Documentation Label Guide.

@github-actions github-actions Bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Sep 7, 2022
@Technoboy- Technoboy- removed the doc-not-needed Your PR changes do not impact docs label Sep 7, 2022
@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Sep 7, 2022
Technoboy-
Technoboy- approved these changes Sep 7, 2022
View reviewed changes

@Technoboy- Technoboy- left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to add some comments for line 809~814

@poorbarcode

poorbarcode commented Sep 7, 2022

Copy link
Copy Markdown
Contributor Author

Hi @Technoboy-

Better to add some comments for line 809~814

Already add code-comment, thanks

@poorbarcode

poorbarcode commented Sep 7, 2022

Copy link
Copy Markdown
Contributor Author

/pulsarbot rerun-failure-checks

2 similar comments
@poorbarcode

poorbarcode commented Sep 7, 2022

Copy link
Copy Markdown
Contributor Author

/pulsarbot rerun-failure-checks

@poorbarcode

poorbarcode commented Sep 7, 2022

Copy link
Copy Markdown
Contributor Author

/pulsarbot rerun-failure-checks

@Technoboy-
Technoboy- merged commit 5c67ded into apache:master Sep 8, 2022
Technoboy- pushed a commit that referenced this pull request Sep 8, 2022
@poorbarcode @Technoboy-
[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRem…
4cef40c 
…oveBundleAndLoad (#17487)
@heesung-sohn

heesung-sohn commented Sep 8, 2022

Copy link
Copy Markdown
Contributor

LGTM

@poorbarcode
poorbarcode deleted the flaky/testModularLoadManagerRemoveBundleAndLoad_ branch September 17, 2022 02:44
@heesung-sohn

heesung-sohn commented Mar 4, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor 
  // Wait until "ModularLoadManager" completes processing the ZK notification.
        ModularLoadManagerWrapper modularLoadManagerWrapper = (ModularLoadManagerWrapper) loadManager;
        ModularLoadManagerImpl modularLoadManager = (ModularLoadManagerImpl) modularLoadManagerWrapper.getLoadManager();
        ScheduledExecutorService scheduler = Whitebox.getInternalState(modularLoadManager, "scheduler");

We don't have the scheduler anymore in ModularLoadManagerImpl. I think this test is failing now, caused by #19656

@heesung-sohn heesung-sohn mentioned this pull request Mar 4, 2023
Merged
5 tasks
@AnonHxy AnonHxy mentioned this pull request Mar 4, 2023
Merged
5 tasks
liangyepianzhou pushed a commit that referenced this pull request Jul 6, 2023
@poorbarcode @liangyepianzhou
[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRem…
44c033c 
…oveBundleAndLoad (#17487)

(cherry picked from commit 5c67ded)
nicoloboschi pushed a commit to datastax/pulsar that referenced this pull request Jul 17, 2023
@poorbarcode @nicoloboschi
[fix][flaky-test]NamespaceServiceTest.flaky/testModularLoadManagerRem…
14b3f0a 
…oveBundleAndLoad (apache#17487)

(cherry picked from commit 5c67ded)
(cherry picked from commit 44c033c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Technoboy- Technoboy- Technoboy- approved these changes
@gaoran10 gaoran10 gaoran10 approved these changes
@Demogorgon314 Demogorgon314 Demogorgon314 approved these changes

Assignees

@poorbarcode poorbarcode

Labels

cherry-picked/branch-2.10 cherry-picked/branch-2.11 doc-not-needed Your PR changes do not impact docs release/2.10.5 release/2.11.1

Projects

None yet

Milestone

3.0.0

Development

Successfully merging this pull request may close these issues.

Flaky-test: NamespaceServiceTest. testModularLoadManagerRemoveBundleAndLoad

7 participants

@poorbarcode @codelipenghui @heesung-sohn @Technoboy- @gaoran10 @Demogorgon314 @liangyepianzhou