Skip to content

refactor: Incorporate RewriteDisjunctivePredicate rule into SimplifyExpressions #13032

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
alamb merged 4 commits into from
Oct 22, 2024
Merged

refactor: Incorporate RewriteDisjunctivePredicate rule into SimplifyExpressions #13032

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
36b81bf
Elliminate common factors in disjunctions
eejbyfeldt Oct 9, 2024
97214eb
Remove RewriteDisjunctivePredicate rule
eejbyfeldt Oct 20, 2024
7ac193f
Fix cse test
eejbyfeldt Oct 21, 2024
876de7e
Update datafusion/optimizer/src/simplify_expressions/expr_simplifier.rs
eejbyfeldt Oct 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix cse test
  • Loading branch information
@eejbyfeldt
eejbyfeldt committed Oct 21, 2024
commit 7ac193fa89507de7dc711948f47437645fd2c1e6
10 changes: 5 additions & 5 deletions datafusion/sqllogictest/test_files/cse.slt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,18 +200,18 @@ physical_plan
query TT
EXPLAIN SELECT
(a = 1 OR random() = 0) AND (a = 2 OR random() = 1) AS c1,
Copy link
Contributor

@alamb alamb Oct 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?

I ran the test locally for the original and here it is

query TT
EXPLAIN SELECT
    (a = 1 OR random() = 0) AND (a = 1 OR random() = 1) AS c1,
    (a = 2 AND random() = 0) OR (a = 2 AND random() = 1) AS c2,
    CASE WHEN a + 3 = 0 THEN a + 3 + random() ELSE 0 END AS c3,
    CASE WHEN a + 4 = 0 THEN 0 ELSE a + 4 + random() END AS c4
FROM t1
----
logical_plan
01)Projection: (__common_expr_1 OR random() = Float64(0)) AND (__common_expr_1 OR random() = Float64(1)) AS c1, t1.a = Float64(2) AND (random() = Float64(0) OR random() = Float64(1)) AS c2, CASE WHEN __common_expr_2 = Float64(0) THEN __common_expr_2 + random() ELSE Float64(0) END AS c3, CASE WHEN __common_expr_3 = Float64(0) THEN Float64(0) ELSE __common_expr_3 + random() END AS c4
02)--Projection: t1.a = Float64(1) AS __common_expr_1, t1.a + Float64(3) AS __common_expr_2, t1.a + Float64(4) AS __common_expr_3, t1.a
03)----TableScan: t1 projection=[a]
physical_plan
01)ProjectionExec: expr=[(__common_expr_1@0 OR random() = 0) AND (__common_expr_1@0 OR random() = 1) as c1, a@3 = 2 AND (random() = 0 OR random() = 1) as c2, CASE WHEN __common_expr_2@1 = 0 THEN __common_expr_2@1 + random() ELSE 0 END as c3, CASE WHEN __common_expr_3@2 = 0 THEN 0 ELSE __common_expr_3@2 + random() END as c4]
02)--ProjectionExec: expr=[a@0 = 1 as __common_expr_1, a@0 + 3 as __common_expr_2, a@0 + 4 as __common_expr_3, a@0 as a]
03)----MemoryExec: partitions=1, partition_sizes=[0]

@peter-toth does this look right to you?

Copy link
Contributor

@peter-toth peter-toth Oct 22, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this change looks good to me to avoid simplification in this CSE test.
(Actually the same trick is applied in the previous test called # Surely only once but also conditionally evaluated expressions a bit above.)

Out of curiosity, volatile common expressions don't get simplified, do they?
E.g. (a = 1 AND random() = 0) OR (a = 2 AND random() = 0) is not rewritten to random() = 0 AND (a = 1 OR a = 2)?

Copy link
Contributor Author

@eejbyfeldt eejbyfeldt Oct 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Actually the same trick is applied in the previous test called # Surely only once but also conditionally evaluated expressions a bit above.)

That was also changed by me in #12746 for basically the same reason for another rule.

Out of curiosity, volatile common expressions don't get simplified, do they?
E.g. (a = 1 AND random() = 0) OR (a = 2 AND random() = 0) is not rewritten to random() = 0 AND (a = 1 OR a = 2)?

It would end up being rewritten. I agree that does not sound correct. It should probably be addressed separatly as the issue applies to many of the other simplification rules. Created #13060

(a = 1 AND random() = 0) OR (a = 2 AND random() = 1) AS c2,
(a = 2 AND random() = 0) OR (a = 1 AND random() = 1) AS c2,
CASE WHEN a + 3 = 0 THEN a + 3 + random() ELSE 0 END AS c3,
CASE WHEN a + 4 = 0 THEN 0 ELSE a + 4 + random() END AS c4
FROM t1
----
logical_plan
01)Projection: (__common_expr_1 OR random() = Float64(0)) AND (t1.a = Float64(2) OR random() = Float64(1)) AS c1, __common_expr_1 AND random() = Float64(0) OR t1.a = Float64(2) AND random() = Float64(1) AS c2, CASE WHEN __common_expr_2 = Float64(0) THEN __common_expr_2 + random() ELSE Float64(0) END AS c3, CASE WHEN __common_expr_3 = Float64(0) THEN Float64(0) ELSE __common_expr_3 + random() END AS c4
02)--Projection: t1.a = Float64(1) AS __common_expr_1, t1.a + Float64(3) AS __common_expr_2, t1.a + Float64(4) AS __common_expr_3, t1.a
01)Projection: (__common_expr_1 OR random() = Float64(0)) AND (__common_expr_2 OR random() = Float64(1)) AS c1, __common_expr_2 AND random() = Float64(0) OR __common_expr_1 AND random() = Float64(1) AS c2, CASE WHEN __common_expr_3 = Float64(0) THEN __common_expr_3 + random() ELSE Float64(0) END AS c3, CASE WHEN __common_expr_4 = Float64(0) THEN Float64(0) ELSE __common_expr_4 + random() END AS c4
02)--Projection: t1.a = Float64(1) AS __common_expr_1, t1.a = Float64(2) AS __common_expr_2, t1.a + Float64(3) AS __common_expr_3, t1.a + Float64(4) AS __common_expr_4
03)----TableScan: t1 projection=[a]
physical_plan
01)ProjectionExec: expr=[(__common_expr_1@0 OR random() = 0) AND (a@3 = 2 OR random() = 1) as c1, __common_expr_1@0 AND random() = 0 OR a@3 = 2 AND random() = 1 as c2, CASE WHEN __common_expr_2@1 = 0 THEN __common_expr_2@1 + random() ELSE 0 END as c3, CASE WHEN __common_expr_3@2 = 0 THEN 0 ELSE __common_expr_3@2 + random() END as c4]
02)--ProjectionExec: expr=[a@0 = 1 as __common_expr_1, a@0 + 3 as __common_expr_2, a@0 + 4 as __common_expr_3, a@0 as a]
01)ProjectionExec: expr=[(__common_expr_1@0 OR random() = 0) AND (__common_expr_2@1 OR random() = 1) as c1, __common_expr_2@1 AND random() = 0 OR __common_expr_1@0 AND random() = 1 as c2, CASE WHEN __common_expr_3@2 = 0 THEN __common_expr_3@2 + random() ELSE 0 END as c3, CASE WHEN __common_expr_4@3 = 0 THEN 0 ELSE __common_expr_4@3 + random() END as c4]
02)--ProjectionExec: expr=[a@0 = 1 as __common_expr_1, a@0 = 2 as __common_expr_2, a@0 + 3 as __common_expr_3, a@0 + 4 as __common_expr_4]
03)----MemoryExec: partitions=1, partition_sizes=[0]

# Only conditionally evaluated expressions
Expand Down