Skip to content

chore: update Arrow patch version#7

Open
discord9 wants to merge 2 commits into
GreptimeTeam:greptimedb-53.1.0-function-signature-exec-errorfrom
discord9:fix/arrow-select-58-3-join-tests
Open

chore: update Arrow patch version#7
discord9 wants to merge 2 commits into
GreptimeTeam:greptimedb-53.1.0-function-signature-exec-errorfrom
discord9:fix/arrow-select-58-3-join-tests

Conversation

@discord9

@discord9 discord9 commented Jun 9, 2026

Copy link
Copy Markdown

Motivation

cargo test --workspace on the Greptime DataFusion 53.1.0 fork exposed debug-mode failures in the physical-plan join tests. The failures are not caused by join logic changes: they all panic inside arrow-select 58.0.0 at src/coalesce/primitive.rs:61:

assertion `left == right` failed
  left: 4
 right: 2

The assertion assumes Vec::reserve(batch_size) makes capacity == batch_size, but Rust only guarantees capacity >= batch_size. For small batch sizes, the allocator can round capacity up to 4, so the debug assertion is invalid even though the coalescing logic is otherwise fine. This was already fixed upstream in later Arrow 58.x patch releases by removing that assertion.

This PR updates the locked Arrow patch versions to 58.3.0 so the fork uses the fixed arrow-select and the join tests no longer fail in debug builds.

Because this lockfile-only PR triggers the Security audit workflow, it also updates two vulnerable transitive crates reported by cargo audit:

  • astral-tokio-tar 0.6.0 -> 0.6.2
  • rustls-webpki 0.103.11 -> 0.103.13

Summary

  • Update locked Arrow crates from 58.0.0 to 58.3.0 for the DataFusion 53.1.0 fork branch
  • Pull in the upstream arrow-select fix that removes the invalid debug assertion after Vec::reserve
  • Fix debug-mode join test failures in datafusion-physical-plan
  • Update vulnerable transitive dependencies needed for the lockfile-triggered audit workflow

Validation

  • cargo audit — no vulnerabilities, only 4 allowed warnings
  • cargo check --profile ci --workspace --all-targets --features integration-tests --locked
  • cargo test -p datafusion-physical-plan --lib -- 'joins::' — 928 passed, 348 filtered out
  • RUST_BACKTRACE=1 cargo test --profile ci --exclude datafusion-examples --exclude ffi_example_table_provider --exclude datafusion-benchmarks --exclude datafusion-cli --workspace --lib --tests --bins --features serde,avro,json,backtrace,integration-tests,parquet_encryption — 7778 passed, 1 ignored
  • git submodule update --init
  • cargo test --workspace — 7879 passed, 1 ignored

Notes

The first full workspace run failed only because Arrow/Parquet test data submodules were missing (ARROW_TEST_DATA / PARQUET_TEST_DATA); after initializing submodules, the full workspace test passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant