backport: Allow dynamic filter pushdown for left join#3
Merged
evenyag merged 2 commits intoJun 9, 2026
Conversation
Signed-off-by: discord9 <discord9@163.com>
5 tasks
waynexia
approved these changes
Jun 9, 2026
d171145
into
GreptimeTeam:greptimedb-53.1.0-function-signature-exec-error
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backport upstream DataFusion's HashJoin dynamic filter support for join types whose probe side is safe for ON-clause filter pushdown.
This follows the original DataFusion feature behavior from apache#20447:
HashJoinExecnow gates self-generated dynamic filters withJoinType::on_lr_is_preserved()instead of a Greptime-specificInner | Leftrule.The dynamic filter is still built from the left/build side join keys and applied to the right/probe side.
Note on null-aware LeftAnti
This backport intentionally keeps the behavior aligned with upstream DataFusion and does not add Greptime-specific handling for null-aware
LeftAnti/NOT INsemantics.In particular, this PR does not introduce an extra
null_awareguard inHashJoinExec::allow_join_dynamic_filter_pushdown(). Any semantic caveats around null-aware anti joins are kept consistent with the original upstream feature and are considered out of scope for this backport.What changed
JoinType::on_lr_is_preserved()todatafusion-common.HashJoinExec::allow_join_dynamic_filter_pushdown()to check whether the right/probe side is preserved for ON-clause pushdown.Why this is safe
The self-generated HashJoin dynamic filter only filters the probe side by build-side join-key values.
For a physical
LEFT JOIN, this is safe because rows on the right/probe side whose keys are not present on the left/build side cannot contribute to the output, while unmatched left rows are still preserved.More generally, this backport keeps the upstream DataFusion rule: enable the feature when the probe side is preserved according to
on_lr_is_preserved().Validation
cargo fmt --allcargo test -p datafusion-physical-plan test_allow_join_dynamic_filter_pushdown_gate -- --nocapturecargo test -p datafusion-physical-plan test_on_lr_is_preserved -- --nocapturecargo test -p datafusion-sqllogictest --test sqllogictests -- dynamic_filter_pushdown_configcargo test -p tests-integration remote_dyn_filter_test -- --nocaptureThe GreptimeDB integration test verifies:
HashJoinExec: mode=CollectLeft, join_type=LeftMergeScanExecreceives non-emptydyn_filters