Skip to content

WIP: Implement join reordering of fact-dimension joins #950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
andygrove wants to merge 32 commits into from
Closed

WIP: Implement join reordering of fact-dimension joins #950

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
4327283
Upgrade to latest DataFusion rev
andygrove Dec 1, 2022
568d6c6
cargo fmt
andygrove Dec 1, 2022
3988fb1
rough out structure of new join reordering rule
andygrove Dec 2, 2022
4997215
implement is_simple_join
andygrove Dec 2, 2022
0b5b70c
Save progress
andygrove Dec 2, 2022
bd9a65b
invert logic
andygrove Dec 2, 2022
4f02c0c
make stats available
andygrove Dec 2, 2022
086491f
detect fact table
andygrove Dec 4, 2022
69b7faf
unnest joins
andygrove Dec 5, 2022
a9c1341
detect fact and dimension tables
andygrove Dec 5, 2022
f38712e
code is runnable end to end
andygrove Dec 5, 2022
4e8089f
fix bug in join unnesting logic
andygrove Dec 5, 2022
2abb409
bug fixes
andygrove Dec 5, 2022
4382182
more tests
andygrove Dec 5, 2022
95fd2e4
more tests
andygrove Dec 5, 2022
99ee5f3
end to end test, remove dummy tables
andygrove Dec 5, 2022
84e36a6
make deterministic
andygrove Dec 5, 2022
437af21
improve filter detection logic
andygrove Dec 5, 2022
308a1d7
docs
andygrove Dec 5, 2022
afba46a
use official release
andygrove Dec 5, 2022
2268ff2
Merge remote-tracking branch 'upstream/main' into datafusion-15.0.0
andygrove Dec 5, 2022
1df3e7f
Merge branch 'datafusion-15.0.0' into join-reorder-poc
andygrove Dec 5, 2022
e1c320c
Error handling
andygrove Dec 5, 2022
4ab93d6
save progress
andygrove Dec 5, 2022
295a1cf
reimplement resolve_table_plan
andygrove Dec 6, 2022
0abbdab
error handling
andygrove Dec 6, 2022
eb63919
add more table sizes:
andygrove Dec 6, 2022
644110b
ignore not-null filters, and fix order of unfiltered dimension tables
andygrove Dec 6, 2022
06f5215
add a comment
andygrove Dec 6, 2022
e175d88
refactor
andygrove Dec 13, 2022
0a2e11c
implement options
andygrove Dec 13, 2022
7b50aca
fix test for aliased relations
andygrove Dec 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
183 changes: 115 additions & 68 deletions dask_planner/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions dask_planner/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ edition = "2021"
rust-version = "1.62"

[dependencies]
arrow = { version = "26.0.0", features = ["prettyprint"] }
arrow = { version = "28.0.0", features = ["prettyprint"] }
async-trait = "0.1.59"
datafusion-common = "14.0.0"
datafusion-expr = "14.0.0"
datafusion-optimizer = "14.0.0"
datafusion-sql = "14.0.0"
datafusion-common = "15.0.0"
datafusion-expr = "15.0.0"
datafusion-optimizer = "15.0.0"
datafusion-sql = "15.0.0"
env_logger = "0.10"
log = "^0.4"
mimalloc = { version = "*", default-features = false }
Expand Down
Loading