Skip to content

WIP: Implement join reordering of fact-dimension joins#950

Closed
andygrove wants to merge 32 commits into
dask-contrib:mainfrom
andygrove:join-reorder-poc
Closed

WIP: Implement join reordering of fact-dimension joins#950
andygrove wants to merge 32 commits into
dask-contrib:mainfrom
andygrove:join-reorder-poc

Conversation

@andygrove

@andygrove andygrove commented Dec 2, 2022

Copy link
Copy Markdown
Contributor

This PR implements a new logical plan optimization rule based on the paper Improving Join Reordering for Large Scale Distributed Computing

Builds on #949

@codecov-commenter

codecov-commenter commented Dec 2, 2022

Copy link
Copy Markdown

Codecov Report

Merging #950 (7b50aca) into main (f3fbdd0) will increase coverage by 1.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #950      +/-   ##
==========================================
+ Coverage   76.62%   77.63%   +1.01%     
==========================================
  Files          73       75       +2     
  Lines        4115     4230     +115     
  Branches      751      770      +19     
==========================================
+ Hits         3153     3284     +131     
+ Misses        792      774      -18     
- Partials      170      172       +2     
Impacted Files Coverage Δ
dask_sql/__init__.py 100.00% <0.00%> (ø)
dask_sql/utils.py 99.03% <0.00%> (ø)
dask_sql/physical/rel/custom/wrappers.py 64.59% <0.00%> (+0.52%) ⬆️
dask_sql/physical/rex/core/literal.py 60.95% <0.00%> (+2.85%) ⬆️
dask_sql/_version.py 35.31% <0.00%> (+3.95%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@andygrove andygrove changed the title WIP: Join Reordering proof-of-concept WIP: Implement join reordering of fact-dimension joins Dec 14, 2022
@andygrove andygrove closed this Jan 13, 2023
sarahyurick added a commit to sarahyurick/dask-sql that referenced this pull request Feb 2, 2023
charlesbluca added a commit that referenced this pull request Apr 5, 2023
* add changes from #950

* add changes from apache/datafusion#4620

* minor changes

* save df 17 progress

* better save

* fix optimize_children logic

* add FIXMEs

* style fix

* remove bracket

* add warning

* address reviews

* Apply suggestions from code review

Co-authored-by: Charles Blackmon-Luca <20627856+charlesbluca@users.noreply.github.com>

* add more suggestions

---------

Co-authored-by: Charles Blackmon-Luca <20627856+charlesbluca@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants