[SPARK-10300] [build] [tests] Add support for test tags in run-tests.py. by vanzin · Pull Request #8437 · apache/spark

vanzin · 2015-08-25T22:25:53Z

This change does two things:

tag a few tests and adds the mechanism in the build to be able to disable those tags,
both in maven and sbt, for both junit and scalatest suites.
add some logic to run-tests.py to disable some tags depending on what files have
changed; that's used to disable expensive tests when a module hasn't explicitly
been changed, to speed up testing for changes that don't directly affect those
modules.

SparkQA · 2015-08-25T22:36:24Z

Test build #41566 has finished for PR 8437 at commit 4e26e43.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class FPGrowthModel[Item: ClassTag] @Since("1.3.0") (
- class FreqItemset[Item] @Since("1.3.0") (
- class FreqSequence[Item] @Since("1.5.0") (
- class PrefixSpanModel[Item] @Since("1.5.0") (

vanzin · 2015-08-25T22:39:59Z

retest this please

SparkQA · 2015-08-25T22:50:47Z

Test build #41567 has finished for PR 8437 at commit 833502f.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

vanzin · 2015-08-25T22:53:39Z

retest this please

…es).

SparkQA · 2015-08-26T01:42:06Z

Test build #41568 has finished for PR 8437 at commit ef737c0.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-08-26T01:52:56Z

Test build #41572 has finished for PR 8437 at commit 3a2979f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Based on squito:SPARK-4746

From squito:SPARK-4746 it seems like they might not even be needed.

…iveTest. This allows a small set of tests to run for every PR, while runnign the whole test suite when changes to sql code are made (as decided by run-tests.py).

vanzin · 2015-08-26T17:37:33Z

@pwendell @marmbrus what do you think of this approach? It's partly based on code that @squito wrote before to add tags for long-running tests. Main change is that I hooked things up to run-tests.py to automate enabling / disabling some tags.

Note that the tests for this PR will run everything (since I'm touching files in the respective modules), but I tested locally that things are disabled properly when they should be.

SparkQA · 2015-08-26T17:49:06Z

Test build #41639 has finished for PR 8437 at commit b7d0507.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

This makes sbt happy now that I put the junit assembly in the shared dependencies in the root pom.

SparkQA · 2015-08-26T18:08:52Z

Test build #41641 has finished for PR 8437 at commit 5c107a5.

This patch fails to build.
This patch merges cleanly.
This patch adds no public classes.

marmbrus · 2015-08-26T18:19:46Z

The ExtendedHiveTest tag looks good to me. I would consider dropping the random smoke test part of things since we have lot of SQL tests and even hive tests that will still be running, so I'm not sure its worth the complexity. I would be okay just marking the whole test as ExtendedHiveTest.

Thanks for working on this!

vanzin · 2015-08-26T18:46:29Z

I would be okay just marking the whole test as ExtendedHiveTest.

Easy enough (good thing I made that change in a single commit).

…xtendedHiveTest." This reverts commit b7d0507.

marmbrus · 2015-08-26T19:14:15Z

SQL changes LGTM. I'll let @pwendell comment on build changes.

Conflicts: dev/run-tests-jenkins

SparkQA · 2015-08-26T21:17:06Z

Test build #41642 has finished for PR 8437 at commit 85fed20.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

vanzin · 2015-08-26T21:24:08Z

pyspark failures, seem unrelated. in any case... retest this please

SparkQA · 2015-08-26T22:11:50Z

Test build #41645 has finished for PR 8437 at commit 83bec7f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-08-26T22:14:05Z

Test build #41648 has finished for PR 8437 at commit 131e658.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

vanzin · 2015-08-26T22:15:22Z

Another pyspark failure (that I've seen countless times in other PRs).

vanzin · 2015-08-27T00:09:00Z

retest this please

SparkQA · 2015-08-27T00:32:36Z

Test build #41649 has finished for PR 8437 at commit 131e658.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-08-27T03:29:23Z

Test build #41657 has finished for PR 8437 at commit 131e658.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2015-08-27T03:37:04Z

Test build #41658 has finished for PR 8437 at commit 090e1d4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

andrewor14 · 2015-09-01T20:00:40Z

@JoshRosen should look at this

vanzin · 2015-09-09T00:59:37Z

@JoshRosen any comments?

vanzin · 2015-09-14T20:14:55Z

No comments? I already got a +1 from Michael, so as far as I can tell, this is good to go.

pwendell · 2015-09-15T05:49:11Z

LGTM - I've modified that script recently enough to be familiar with how it works. This seems like a good approach and could be useful for us in the future.

vanzin · 2015-09-15T17:43:13Z

ok, merging.

vanzin · 2015-09-15T19:58:35Z

This seems to have caused PR tests to start failing... I'll try things locally again and revert the patch if needed.

mengxr · 2015-09-15T20:01:25Z

Shall we revert the PR first? It is blocking other PRs.

vanzin · 2015-09-15T20:04:14Z

Done.

vanzin · 2015-09-15T20:54:37Z

For the curious, the issue is that the junit test runners (both surefire and the sbt one) need the tag classes around or they'll throw an error, and that happens when building modules where the tag is not available. I'm thinking about a workaround that does not involve removing the junit functionality.

JoshRosen · 2015-09-30T23:47:37Z

Are there plans to revive this PR and attempt another merge?

vanzin · 2015-10-01T13:57:06Z

@JoshRosen #8775

WIP DO NOT MERGE. Testing out test tags.

4e26e43

vanzin force-pushed the test-tags branch from 2f59ee2 to 833502f Compare August 25, 2015 22:38

Oh great now we have different styles depending on language...

ef737c0

vanzin force-pushed the test-tags branch from 833502f to ef737c0 Compare August 25, 2015 22:53

Always run all tests locally (since code doesn't detect changed modul…

3a2979f

…es).

vanzin force-pushed the test-tags branch from 3585998 to 3a2979f Compare August 25, 2015 23:16

Marcelo Vanzin added 5 commits August 25, 2015 20:01

Add a tag for yarn integration tests.

a02c07a

Add tag exclusion to junit also.

05304a6

Based on squito:SPARK-4746

Remove scalatest object tags since they're not used.

ce92266

From squito:SPARK-4746 it seems like they might not even be needed.

Declare common test dependencies in root pom only.

27749cf

Tag a random sample of tests in HiveCompatibilitySuite with ExtendedH…

b7d0507

…iveTest. This allows a small set of tests to run for every PR, while runnign the whole test suite when changes to sql code are made (as decided by run-tests.py).

Marcelo Vanzin added 2 commits August 26, 2015 10:52

Workaround for an ivy issue?

5c107a5

Use same junit version for maven and sbt.

85fed20

This makes sbt happy now that I put the junit assembly in the shared dependencies in the root pom.

Revert "Tag a random sample of tests in HiveCompatibilitySuite with E…

83bec7f

…xtendedHiveTest." This reverts commit b7d0507.

vanzin changed the title ~~WIP DO NOT MERGE. Testing out test tags.~~ [SPARK-10300] [build] [tests] Add support for test tags in run-test.py. Aug 26, 2015

vanzin changed the title ~~[SPARK-10300] [build] [tests] Add support for test tags in run-test.py.~~ [SPARK-10300] [build] [tests] Add support for test tags in run-tests.py. Aug 26, 2015

Merge branch 'master' into test-tags

131e658

Conflicts: dev/run-tests-jenkins

Define sbt_test_goals for yarn module.

090e1d4

vanzin mentioned this pull request Aug 31, 2015

[SPARK-10378][SQL][Test] Remove HashJoinCompatibilitySuite. #8542

Closed

asfgit closed this in 8abef21 Sep 15, 2015

vanzin deleted the test-tags branch September 15, 2015 17:50

Uh oh!

Conversation

vanzin commented Aug 25, 2015

Uh oh!

SparkQA commented Aug 25, 2015

Uh oh!

vanzin commented Aug 25, 2015

Uh oh!

SparkQA commented Aug 25, 2015

Uh oh!

vanzin commented Aug 25, 2015

Uh oh!

SparkQA commented Aug 26, 2015

Uh oh!

SparkQA commented Aug 26, 2015

Uh oh!

vanzin commented Aug 26, 2015

Uh oh!

SparkQA commented Aug 26, 2015

Uh oh!

SparkQA commented Aug 26, 2015

Uh oh!

marmbrus commented Aug 26, 2015

Uh oh!

vanzin commented Aug 26, 2015

Uh oh!

marmbrus commented Aug 26, 2015

Uh oh!

SparkQA commented Aug 26, 2015

Uh oh!

vanzin commented Aug 26, 2015

Uh oh!

SparkQA commented Aug 26, 2015

Uh oh!

SparkQA commented Aug 26, 2015

Uh oh!

vanzin commented Aug 26, 2015

Uh oh!

vanzin commented Aug 27, 2015

Uh oh!

SparkQA commented Aug 27, 2015

Uh oh!

SparkQA commented Aug 27, 2015

Uh oh!

SparkQA commented Aug 27, 2015

Uh oh!

andrewor14 commented Sep 1, 2015

Uh oh!

vanzin commented Sep 9, 2015

Uh oh!

vanzin commented Sep 14, 2015

Uh oh!

pwendell commented Sep 15, 2015

Uh oh!

vanzin commented Sep 15, 2015

Uh oh!

vanzin commented Sep 15, 2015

Uh oh!

mengxr commented Sep 15, 2015

Uh oh!

vanzin commented Sep 15, 2015

Uh oh!

vanzin commented Sep 15, 2015

Uh oh!

JoshRosen commented Sep 30, 2015

Uh oh!

vanzin commented Oct 1, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants