Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
cd6433c
Use BatchCoaleser in sort merge join instead of calling coalesce_batc…
mbutrovich Nov 21, 2025
d29fd29
Merge branch 'main' into smj
mbutrovich Nov 21, 2025
c1b58b9
Merge branch 'main' into smj
mbutrovich Nov 25, 2025
a655212
stash
mbutrovich Nov 25, 2025
4ed5cd4
Stash with assertions.
mbutrovich Nov 25, 2025
4364656
Stash with assertions.
mbutrovich Nov 25, 2025
7a41fe6
encapsulate
mbutrovich Nov 25, 2025
b986fd7
encapsulate
mbutrovich Nov 25, 2025
387c882
encapsulate
mbutrovich Dec 1, 2025
efa2996
pre-refactor
mbutrovich Dec 1, 2025
a5c926f
get rid of confusing output_size
mbutrovich Dec 1, 2025
f725308
refactor
mbutrovich Dec 1, 2025
4cc21e8
refactor
mbutrovich Dec 1, 2025
f6430db
fix double concat for filtered joins
mbutrovich Dec 1, 2025
32021cb
more elided concats
mbutrovich Dec 2, 2025
2e0f211
remove dead code
mbutrovich Dec 2, 2025
37bb875
passes
mbutrovich Dec 2, 2025
2ac80f6
Merge branch 'main' into smj5
mbutrovich Dec 2, 2025
8c69056
comments
mbutrovich Dec 2, 2025
67877e6
clippy, comments
mbutrovich Dec 2, 2025
e7b94e5
Remove unused import
mbutrovich Dec 2, 2025
7c55ad9
optimize concat_batches call
mbutrovich Dec 2, 2025
ad583d2
Merge branch 'main' into smj
mbutrovich Dec 2, 2025
43a945f
fix metrics collection filtered joins
mbutrovich Dec 2, 2025
6a4e664
pass through batches that are batch_size / 2 similar to LimitedBatchC…
mbutrovich Dec 3, 2025
36a73e5
Merge branch 'main' into smj
mbutrovich Dec 3, 2025
1000afa
Merge branch 'main' into smj
mbutrovich Dec 4, 2025
66ea027
Address PR feedback.
mbutrovich Dec 4, 2025
eb5637e
Merge branch 'main' into smj
mbutrovich Dec 4, 2025
86cbc5c
Remove stray import.
mbutrovich Dec 4, 2025
2ce09f1
Merge branch 'main' into smj
mbutrovich Dec 8, 2025
8ec9e92
We're spending a ton of time resizing in append_output_pair. Try to h…
mbutrovich Dec 8, 2025
2389fd6
Merge branch 'main' into smj
mbutrovich Dec 8, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Stash with assertions.
  • Loading branch information
mbutrovich committed Nov 25, 2025
commit 43646568b9a8e2ae33f4882f970bb70d132d4adc
5 changes: 5 additions & 0 deletions datafusion/physical-plan/src/joins/sort_merge_join/stream.rs
Original file line number Diff line number Diff line change
Expand Up @@ -410,6 +410,11 @@ impl JoinedRecordBatches {
}

fn clear(&mut self) {
// Note: clear() can be called when batches still contains data!
// This happens in filter_joined_batch() after concat_batches() has read
// the batches but before they're removed. The batches have been processed
// into output, so clearing them here is the final cleanup step.

self.batches.clear();
self.batch_ids.clear();
self.filter_mask = BooleanBuilder::new();
Expand Down