ARROW-9438: [CI] Add spark patch to compile with recent Arrow Java changes#7746
ARROW-9438: [CI] Add spark patch to compile with recent Arrow Java changes#7746BryanCutler wants to merge 2 commits into
Conversation
|
@ursabot crossbow submit test-conda-python-3.8-spark-master |
|
AMD64 Conda Crossbow Submit (#119311) builder has been succeeded. Revision: 5ec6979 Submitted crossbow builds: ursa-labs/crossbow @ ursabot-574
|
|
This should fix the issues with Spark Scala tests, but there will most likely still be a failure with PySpark grouped map test with structs of timestamps. That will need a bit more effort and another patch cc @kszucs |
|
@ursabot crossbow submit test-conda-python-3.8-spark-master |
|
AMD64 Conda Crossbow Submit (#119359) builder has been succeeded. Revision: d1ed900 Submitted crossbow builds: ursa-labs/crossbow @ ursabot-575
|
|
I should have been aware of the effects ARROW-9300 was goign to have downstream. Anything I can assist with @BryanCutler ? |
|
Timezone related failure from crossbow https://github.com/ursa-labs/crossbow/runs/868904910#step:6:14426 |
|
AFAICT this can be merged? The timestamp related problems will require follow up work |
|
Thanks @kszucs for fixing up the Docker file!
No worries @rymurr ! The java changes were pretty minimal and I think that's all working well now.
The current timezone failure is the result of #7604 and was expected here. Since this improves a little to only 1 known failure, I think this is ok to be merged. I will open another PR for the remaining issue when I'm able to get to it. |
6 participants
Recent changes in Arrow Java from ARROW-9300 now require adding a dependency on arrow-memory-netty to provide a default allocator. This adds a patch to build spark with the required dependency.