chore: update Arrow patch version#7
Open
discord9 wants to merge 2 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
cargo test --workspaceon the Greptime DataFusion 53.1.0 fork exposed debug-mode failures in the physical-plan join tests. The failures are not caused by join logic changes: they all panic insidearrow-select 58.0.0atsrc/coalesce/primitive.rs:61:The assertion assumes
Vec::reserve(batch_size)makescapacity == batch_size, but Rust only guaranteescapacity >= batch_size. For small batch sizes, the allocator can round capacity up to 4, so the debug assertion is invalid even though the coalescing logic is otherwise fine. This was already fixed upstream in later Arrow 58.x patch releases by removing that assertion.This PR updates the locked Arrow patch versions to 58.3.0 so the fork uses the fixed
arrow-selectand the join tests no longer fail in debug builds.Because this lockfile-only PR triggers the Security audit workflow, it also updates two vulnerable transitive crates reported by
cargo audit:astral-tokio-tar0.6.0 -> 0.6.2rustls-webpki0.103.11 -> 0.103.13Summary
arrow-selectfix that removes the invalid debug assertion afterVec::reservedatafusion-physical-planValidation
cargo audit— no vulnerabilities, only 4 allowed warningscargo check --profile ci --workspace --all-targets --features integration-tests --lockedcargo test -p datafusion-physical-plan --lib -- 'joins::'— 928 passed, 348 filtered outRUST_BACKTRACE=1 cargo test --profile ci --exclude datafusion-examples --exclude ffi_example_table_provider --exclude datafusion-benchmarks --exclude datafusion-cli --workspace --lib --tests --bins --features serde,avro,json,backtrace,integration-tests,parquet_encryption— 7778 passed, 1 ignoredgit submodule update --initcargo test --workspace— 7879 passed, 1 ignoredNotes
The first full workspace run failed only because Arrow/Parquet test data submodules were missing (
ARROW_TEST_DATA/PARQUET_TEST_DATA); after initializing submodules, the full workspace test passed.