Skip to content

Set up build pipeline for Spark-3.0.0#348

Merged
imback82 merged 20 commits into
dotnet:masterfrom
imback82:build_3.0
Sep 8, 2020
Merged

Set up build pipeline for Spark-3.0.0#348
imback82 merged 20 commits into
dotnet:masterfrom
imback82:build_3.0

Conversation

@imback82

@imback82 imback82 commented Nov 22, 2019

Copy link
Copy Markdown
Contributor

This PR sets up CI for Spark-3.0.0.

There are few failing tests that were disabled and will be tracked by #657.

@imback82

imback82 commented Nov 22, 2019

Copy link
Copy Markdown
Contributor Author

Currently, the following E2E tests are failing:

  • DataFrameTests/TestGroupedMapUdf (disabled)
  • DataFrameTests/TestSignaturesV2_3_X
  • FunctionsTests/TestSignaturesV2_3_X
  • FunctionsTests/TestSignaturesV2_4_X
  • WindowsSpecTests
  • WindowsTests

Contributions for fixing these are welcome!

@imback82 imback82 added the enhancement New feature or request label Nov 22, 2019
@imback82 imback82 self-assigned this Nov 22, 2019
@imback82

Copy link
Copy Markdown
Contributor Author

I am working on:

  • FunctionsTests/TestSignaturesV2_3_X
  • FunctionsTests/TestSignaturesV2_4_X

@elvaliuliuliu

Copy link
Copy Markdown
Contributor

I will work on:

DataFrameTests/TestGroupedMapUdf
DataFrameTests/TestSignaturesV2_3_X

Comment thread azure-pipelines.yml Outdated
Comment thread script/download-spark-distros.cmd Outdated
@imback82 imback82 mentioned this pull request Sep 6, 2020
6 tasks

// Strip anything below version number.
// For example, "3.0.0-preview" should become "3.0.0".
Version = new Version(versionStr.Split('-')[0]);

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is technically not needed, but I didn't remove this b/c it could be useful when testing preview versions.

}
}

public sealed class SkipIfSparkVersionIsGreaterOrEqualTo : FactAttribute

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is taken from #653.

@imback82 imback82 requested a review from suhsteve September 7, 2020 01:31
@imback82 imback82 added the infrastructure Build pipeline, etc. label Sep 7, 2020
@imback82 imback82 added this to the 1.0.0 milestone Sep 7, 2020
@imback82

imback82 commented Sep 8, 2020

Copy link
Copy Markdown
Contributor Author

Merging this to master. Feel free to leave any comments if you haven't reviewed. cc: @suhsteve @Niharikadutta @AFFogarty @elvaliuliuliu

@imback82 imback82 merged commit aed9214 into dotnet:master Sep 8, 2020
@suhsteve

suhsteve commented Sep 8, 2020

Copy link
Copy Markdown
Member

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request infrastructure Build pipeline, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants