Skip to content

CI Fixes#152

Merged
xhochy merged 1 commit intodata-engineering-collective:mainfrom
IzerOnadimQC:ci-failures
May 10, 2024
Merged

CI Fixes#152
xhochy merged 1 commit intodata-engineering-collective:mainfrom
IzerOnadimQC:ci-failures

Conversation

@IzerOnadimQC
Copy link
Copy Markdown
Collaborator

@IzerOnadimQC IzerOnadimQC commented May 9, 2024

CI failures are caused by the following package incompatibilities.

  1. Incompatibility with python 3.11.9 dask/dask#11038 Some versions of dask are incompatible with python 3.11.9. We were accidentally upgrading to this python version due to a --no-py-pin I added that we no longer need. Removing this fixes the issue, but this is not a true fix - perhaps we should pin dask to avoid the problematic versions (I don't know which versions are affected, only that the fix went into 2024.4.1, and has been back-ported to 2024.2.1).
  2. Pyarrow uses pandas.core.internals.DatetimeTZBlock, which will be removed in the next major release, and is already gone from the nightly builds. This is already handled in newer versions of pyarrow by checking the pandas API version, but causes failures for older versions of pyarrow. Perhaps this could be fixed by patching the repodata for pyarrow, but I'm uncertain whether this can be done for an optional dependency like pandas. For now, I've fixed this by testing the pyarrow and pandas nightly builds together, but an actual solution might be to drop support for older version of pyarrow when pandas 3 is released.

@IzerOnadimQC IzerOnadimQC marked this pull request as ready for review May 9, 2024 12:27
@xhochy
Copy link
Copy Markdown
Member

xhochy commented May 10, 2024

You can also repodata patch optional dependencies via constraints in conda-forge.

@xhochy xhochy merged commit 6e3d0cd into data-engineering-collective:main May 10, 2024
@IzerOnadimQC IzerOnadimQC deleted the ci-failures branch May 12, 2024 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants