Skip to content

[SPARK-24573][INFRA] Runs SBT checkstyle after the build to work around a side-effect#21579

Closed
HyukjinKwon wants to merge 10 commits into
apache:masterfrom
HyukjinKwon:investigate-javastyle
Closed

[SPARK-24573][INFRA] Runs SBT checkstyle after the build to work around a side-effect#21579
HyukjinKwon wants to merge 10 commits into
apache:masterfrom
HyukjinKwon:investigate-javastyle

Conversation

@HyukjinKwon

@HyukjinKwon HyukjinKwon commented Jun 17, 2018 •
edited
Loading

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

Seems checkstyle affects the build in the PR builder in Jenkins. I can't reproduce in my local and seems it can only be reproduced in the PR builder.

I was checking the places it goes through and this is just a speculation that checkstyle's compilation in SBT has a side effect to the assembly build.

This PR proposes to run the SBT checkstyle after the build.

How was this patch tested?

Jenkins tests.

@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91979 has started for PR 21579 at commit 8307cbd.

@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91980 has finished for PR 21579 at commit 96f5458.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91981 has started for PR 21579 at commit 01a9bc8.

@HyukjinKwon HyukjinKwon changed the title [DO-NOT-MERGE] Investigate SBT Java stylecheck affecting the build [SPARK-24573][INFRA] Delays SBT checkstyle after the build to work around a side-effect Jun 17, 2018
@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91986 has finished for PR 21579 at commit 7f46304.

  • This patch fails Java style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

HyukjinKwon
HyukjinKwon commented Jun 17, 2018
View reviewed changes
Comment thread dev/run-tests.py
exec_sbt(profiles_and_goals)

if checkstyle:
run_java_style_checks()

@HyukjinKwon HyukjinKwon Jun 17, 2018 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This takes 2-ish mins now given my past tries.

@HyukjinKwon

HyukjinKwon commented Jun 17, 2018

Copy link
Copy Markdown
Member Author

Build / checkstyle passed in https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91981/console

@HyukjinKwon

HyukjinKwon commented Jun 17, 2018

Copy link
Copy Markdown
Member Author

cc @cloud-fan

@HyukjinKwon HyukjinKwon changed the title [SPARK-24573][INFRA] Delays SBT checkstyle after the build to work around a side-effect [SPARK-24573][INFRA] Runs SBT checkstyle after the build to work around a side-effect Jun 17, 2018
@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91988 has started for PR 21579 at commit 6529622.

@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91989 has started for PR 21579 at commit 73a494e.

@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91987 has finished for PR 21579 at commit 14351d6.

  • This patch fails to generate documentation.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon

HyukjinKwon commented Jun 17, 2018

Copy link
Copy Markdown
Member Author

^ I just double checked if it affects Unidoc too which we run after the build.

@viirya

viirya commented Jun 17, 2018

Copy link
Copy Markdown
Member

Any idea why this issue suddenly happens?

@HyukjinKwon

HyukjinKwon commented Jun 17, 2018 •
edited
Loading

Copy link
Copy Markdown
Member Author

One module was recently added and I was suspecting that it caused a effect from compilation by checkstyle to the build. I am not fully sure but I just suspected this by reading related places ... seems at least it fixes the issue though .. Will take it back for now actually. I think I am not fully sure why it suddenly happened.

@HyukjinKwon

HyukjinKwon commented Jun 17, 2018 •
edited
Loading

Copy link
Copy Markdown
Member Author

Other builds above that don't have the results were aborted intentionally. The last two build currently running are:

  1. Current proposal having a Java change to make sure this PR fixes it (7eba44a).
  2. Reverted the Java change by 1 (d902b10).

@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91997 has finished for PR 21579 at commit d902b10.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon

HyukjinKwon commented Jun 17, 2018

Copy link
Copy Markdown
Member Author

retest this please

@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91995 has finished for PR 21579 at commit 7eba44a.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Jun 17, 2018

Copy link
Copy Markdown

Test build #91999 has finished for PR 21579 at commit d902b10.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@viirya

viirya commented Jun 18, 2018

Copy link
Copy Markdown
Member

ping @cloud-fan Shall we merge this to unblock the testing of other PRs?

@HyukjinKwon

HyukjinKwon commented Jun 18, 2018

Copy link
Copy Markdown
Member Author

Let me merge this one. We can revert this at any moment if we see other problems and go another root, for example, clean before assembly.

@HyukjinKwon

HyukjinKwon commented Jun 18, 2018

Copy link
Copy Markdown
Member Author

Or, please revert this and propose another change if anyone is clear on this. I don't mind.

Merged to master.

@HyukjinKwon

HyukjinKwon commented Jun 18, 2018

Copy link
Copy Markdown
Member Author

Thanks for reviewing this @viirya and @felixcheung.

@asfgit asfgit closed this in b0a9352 Jun 18, 2018
@GulajavaMinistudio GulajavaMinistudio mentioned this pull request Jun 18, 2018
Merged
@cloud-fan

cloud-fan commented Jun 18, 2018

Copy link
Copy Markdown
Contributor

thanks for the fix!

@HyukjinKwon HyukjinKwon deleted the investigate-javastyle branch October 16, 2018 12:43
@HyukjinKwon HyukjinKwon mentioned this pull request Dec 9, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@felixcheung felixcheung felixcheung approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@HyukjinKwon @SparkQA @viirya @cloud-fan @felixcheung