ARROW-11086: [Rust] Extend take implementation to more index types#9057
ARROW-11086: [Rust] Extend take implementation to more index types#9057Dandandan wants to merge 2 commits into
Conversation
|
The full set of Rust CI tests did not run on this PR :( Can you please rebase this PR against apache/master to pick up the changes in #9056 so that they do? I apologize for the inconvenience. |
Dandandan
force-pushed
the
take_index
branch
from
e644413 to
c0af0d9
Compare
|
Rebased |
Codecov Report
@@ Coverage Diff @@
## master #9057 +/- ##
=======================================
Coverage 82.61% 82.61%
=======================================
Files 203 204 +1
Lines 50140 50140
=======================================
Hits 41422 41422
Misses 8718 8718
Continue to review full report at Codecov.
|
|
The clippy failures in https://github.com/apache/arrow/pull/9057/checks?check_run_id=1630788725 seem unrelated to your change -- let me check that out... |
|
Ah, I hadn't yet seen #9061 which appears to fix the clippy errors -- thanks @Dandandan ! |
|
FYI @jorgecarleitao @andygrove this (small) PR is needed to finish #9070 |
|
Clippy missing xD |
|
@jorgecarleitao restarted the CI 😄 |
5 participants
Context
The context of this PR is that I want to experiment with a simplified implementation of the hash join in DataFusion which directly can index the build-side array instead of keeping a list of batches. This array could grow beyond 2 ^ 32 billion elements, so would need indexes of type
UInt64rather thanUInt32.Implementation
In the PR I just extend the public
taketo take anyIndexTypewhich implementsArrowNumericTypeandToPrimitive.I am not sure about the consideration before to restrict
taketo onlyUInt32Array.