Skip to content

KAFKA-16888 Fix failed StorageToolTest.testFormatSucceedsIfAllDirectoriesAreAvailable and StorageToolTest.testFormatEmptyDirectory#16186

Merged
chia7712 merged 2 commits intoapache:trunkfrom
brandboat:KAFKA-16888
Jun 4, 2024
Merged

Conversation

@brandboat
Copy link
Copy Markdown
Member

@brandboat brandboat commented Jun 4, 2024

a quick fix for https://issues.apache.org/jira/browse/KAFKA-16888

459da47#diff-4bacfdbf0e63a4d5f3deb1a0d39037a18510ac24ee5ec276fe70bc818ba4d209L505 put extra information as follows to the beginning of the output which break some tests in StorageToolTest that rely on the original output.

metaPropertiesEnsemble=MetaPropertiesEnsemble(metadataLogDir=Optional.empty, dirs={/tmp/kafka-10468059960007009962: EMPTY, /tmp/kafka-11677474438939329103: EMPTY, /tmp/kafka-17616780619664597434: EMPTY})

This pr modified the test testFormatSucceedsIfAllDirectoriesAreAvailable, testFormatEmptyDirectory to get rid of those extra infos to avoid test failed.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

…riesAreAvailable and StorageToolTest.testFormatEmptyDirectory
Copy link
Copy Markdown
Member

@showuon showuon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @brandboat ! I assume the patch fixes the broken tests, but could you please add some info in the PR description for reviewers know why this fixes the issue. Something like:

  1. Why it is broken
  2. In which commit changes the behavior and why changes
  3. How do you fix it

Thanks.

Copy link
Copy Markdown
Member

@chia7712 chia7712 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brandboat thanks for this hot-fix!

val actual = stream.toString().split("\\r?\\n")
val actual = stream.toString().split("\\r?\\n").drop(1)
val expect = availableDirs.map("Formatting %s".format(_))
assertEquals(availableDirs.size, actual.size)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems to me this (failed) check can be removed. All we wan to test is the folders get formatted only once, and that is verified in the following assert.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've addressed comments in the latest commit, thanks for reviewing.

assertEquals(0, StorageTool.
formatCommand(new PrintStream(stream), Seq(tempDir.toString), metaProperties, bootstrapMetadata, MetadataVersion.latestTesting(), ignoreFormatted = false))
assertTrue(stream.toString().startsWith("Formatting %s".format(tempDir)))
assertTrue(stream.toString().split("\\r?\\n").drop(1)(0).startsWith("Formatting %s".format(tempDir)))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can check that output contains the expected content. for example:

assertTrue(stream.toString().split("\\r?\\n").exists(_.startsWith("Formatting %s".format(tempDir))))

drop(1)(0) is too magic to understand to me :)

assertEquals(0, StorageTool.
formatCommand(new PrintStream(stream2), Seq(tempDir.toString), metaProperties, bootstrapMetadata, MetadataVersion.latestTesting(), ignoreFormatted = true))
assertEquals("All of the log directories are already formatted.%n".format(), stream2.toString())
assertEquals("All of the log directories are already formatted.".format(), stream2.toString().split("\\r?\\n").drop(1)(0))
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@brandboat
Copy link
Copy Markdown
Member Author

brandboat commented Jun 4, 2024

Thanks for the fix @brandboat ! I assume the patch fixes the broken tests, but could you please add some info in the PR description for reviewers know why this fixes the issue. Something like:

Why it is broken
In which commit changes the behavior and why changes
How do you fix it
Thanks.

Added some description in pr, thanks ! 😃

Copy link
Copy Markdown
Member

@showuon showuon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Copy Markdown
Member

@chia7712 chia7712 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@chia7712 chia7712 merged commit a08db65 into apache:trunk Jun 4, 2024
@brandboat brandboat deleted the KAFKA-16888 branch June 4, 2024 14:00
chia7712 pushed a commit that referenced this pull request Jun 5, 2024
…riesAreAvailable and StorageToolTest.testFormatEmptyDirectory (#16186)

Reviewers: Luke Chen <showuon@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
TaiJuWu pushed a commit to TaiJuWu/kafka that referenced this pull request Jun 8, 2024
…riesAreAvailable and StorageToolTest.testFormatEmptyDirectory (apache#16186)

Reviewers: Luke Chen <showuon@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
gongxuanzhang pushed a commit to gongxuanzhang/kafka that referenced this pull request Jun 12, 2024
…riesAreAvailable and StorageToolTest.testFormatEmptyDirectory (apache#16186)

Reviewers: Luke Chen <showuon@gmail.com>, Chia-Ping Tsai <chia7712@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants