Skip to content

[SPARK-32657][K8S] Update the log strings we check for & imports in decommission K8s#29479

Closed
holdenk wants to merge 2 commits into
apache:masterfrom
holdenk:SPARK-32657-Decommissioning-tests-update-log-string
Closed

[SPARK-32657][K8S] Update the log strings we check for & imports in decommission K8s#29479
holdenk wants to merge 2 commits into
apache:masterfrom
holdenk:SPARK-32657-Decommissioning-tests-update-log-string

Conversation

@holdenk

@holdenk holdenk commented Aug 19, 2020

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Update the log strings to match the new log messages.

Why are the changes needed?

Tests are failing

Does this PR introduce any user-facing change?

No, test only change.

How was this patch tested?

WIP: Make sure the DecommissionSuite passes in Jenkins.

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Test build #127655 has finished for PR 29479 at commit 4532e5d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32278/

@holdenk

holdenk commented Aug 19, 2020

Copy link
Copy Markdown
Contributor Author

cc @dongjoon-hyun

@dongjoon-hyun

dongjoon-hyun commented Aug 19, 2020

Copy link
Copy Markdown
Member

Thank you, @holdenk .

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32278/

dongjoon-hyun
dongjoon-hyun approved these changes Aug 19, 2020
View reviewed changes

@dongjoon-hyun dongjoon-hyun left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM (Pending K8s IT)

@dongjoon-hyun

dongjoon-hyun commented Aug 19, 2020

Copy link
Copy Markdown
Member

Oh, does it fail?

@holdenk

holdenk commented Aug 19, 2020

Copy link
Copy Markdown
Contributor Author

Huh this is weird that's not something I was expecting ( [ERROR] [Error] /home/jenkins/workspace/SparkPullRequestBuilder-K8s/resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/DecommissionSuite.scala:29: value WORKER_DECOMMISSION_ENABLED is not a member of object org.apache.spark.internal.config.Worker ). Looks like something else might have changed in master since when I started. Let me poke at that real quickly.

@holdenk

holdenk commented Aug 19, 2020

Copy link
Copy Markdown
Contributor Author

Looks like 3092527f7557b64ff9a5bedadfac8bb2f189a9b4 was committed and this broke the test suite. We really need to enable the K8s PRB in core.

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Test build #127658 has finished for PR 29479 at commit 8a88340.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32281/

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32281/

@holdenk

holdenk commented Aug 19, 2020

Copy link
Copy Markdown
Contributor Author

Token failure is probably unrelated, let’s see if it’s transient. Jenkins retest this please

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Test build #127661 has finished for PR 29479 at commit 8a88340.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32284/

@SparkQA

SparkQA commented Aug 19, 2020

Copy link
Copy Markdown

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/32284/

@holdenk holdenk mentioned this pull request Aug 19, 2020
Closed
@holdenk holdenk changed the title [WIP][SPARK-32657][K8S] Update the log strings we check for in the decommission K8s [SPARK-32657][K8S] Update the log strings we check for in the decommission K8s Aug 19, 2020
@holdenk holdenk changed the title [SPARK-32657][K8S] Update the log strings we check for in the decommission K8s [SPARK-32657][K8S] Update the log strings we check for & imports in decommission K8s Aug 20, 2020
agrawaldevesh
agrawaldevesh approved these changes Aug 20, 2020
View reviewed changes

@agrawaldevesh agrawaldevesh left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks !.

@agrawaldevesh

agrawaldevesh commented Aug 20, 2020

Copy link
Copy Markdown

Lets get this in soon, so that it can unblock other PRs.

@dongjoon-hyun

dongjoon-hyun commented Aug 20, 2020

Copy link
Copy Markdown
Member

Merged to master. Thank you, @holdenk and @agrawaldevesh .

@holdenk

holdenk commented Aug 20, 2020

Copy link
Copy Markdown
Contributor Author

Thanks @dongjoon-hyun for merging and calling my attention to the issue last night :)

holdenk added a commit to holdenk/spark that referenced this pull request Oct 27, 2020
@holdenk @dongjoon-hyun
[SPARK-32657][K8S] Update the log strings we check for & imports in d…
6a439c6 
…ecommission K8s

### What changes were proposed in this pull request?

Update the log strings to match the new log messages.

### Why are the changes needed?

Tests are failing

### Does this PR introduce _any_ user-facing change?

No, test only change.

### How was this patch tested?
WIP: Make sure the DecommissionSuite passes in Jenkins.

Closes apache#29479 from holdenk/SPARK-32657-Decommissioning-tests-update-log-string.

Authored-by: Holden Karau <hkarau@apple.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dongjoon-hyun dongjoon-hyun dongjoon-hyun approved these changes
+1 more reviewer
@agrawaldevesh agrawaldevesh agrawaldevesh approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

KUBERNETES

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@holdenk @SparkQA @dongjoon-hyun @agrawaldevesh