Generalize patching to support vendoring#39
Merged
rapids-bot[bot] merged 16 commits intorapidsai:branch-24.06from Apr 4, 2024
Merged
Generalize patching to support vendoring#39rapids-bot[bot] merged 16 commits intorapidsai:branch-24.06from
rapids-bot[bot] merged 16 commits intorapidsai:branch-24.06from
Conversation
…ors to work on empty directories.
…ssor.py Co-authored-by: Richard (Rick) Zamora <rzamora217@gmail.com>
vyasr
added
feature request
Contributor
Author
|
For posterity, this is a redo of #38 because that PR was merged without squashing. |
Member
|
/merge |
vyasr
added a commit
to vyasr/rapids-dask-dependency
that referenced
this pull request
Apr 4, 2024
This PR makes a number of significant changes to the patching infrastructure: 1. This PR reorganizes the patching logic to be based on a more principled approach. Rather than maintaining lists of patch functions that are each responsible for filtering modules to apply themselves to, patches are organized in the patches directory in a tree structure matching dask itself. Patches are found and run by importing the same relative paths within the `patches` directory corresponding to a particular dask or distributed module. 2. It adds proper support for patching submodules. Previously the loader was being disabled whenever a real dask module was being imported, but this is problematic because if some dask modules import others they will pre-populate `sys.modules` with the real modules and therefore the loader will never be used for loading a patched version of the submodule. 3. Patches are no longer just functions applied to modules, they are arbitrary functions executed when a module is imported. As a result, a wider range of modifications is possible than was previously allowed. In particular: 4. The more general functions allow the entire module import to be hijacked and redirected to other modules. 5. The new framework is used to vendor a patched version of the accessor.py module in dask, which resolves the issues observed in dask/dask#11035. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Richard (Rick) Zamora (https://github.com/rjzamora) URL: rapidsai#39
raydouglass
pushed a commit
that referenced
this pull request
Apr 5, 2024
rjzamora
added a commit
to rjzamora/rapids-dask-dependency
that referenced
this pull request
Apr 5, 2024
This reverts commit a529757.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR makes a number of significant changes to the patching infrastructure:
patchesdirectory corresponding to a particular dask or distributed module.sys.moduleswith the real modules and therefore the loader will never be used for loading a patched version of the submodule.dask.dataframeimport error for Python 3.11.9 dask/dask#11035.