Skip to content

[SPARK-37557][SQL] Replace object hash with sort aggregate if child is already sorted#34824

Closed
c21 wants to merge 1 commit into
apache:masterfrom
c21:agg-rule-followup
Closed

[SPARK-37557][SQL] Replace object hash with sort aggregate if child is already sorted#34824
c21 wants to merge 1 commit into
apache:masterfrom
c21:agg-rule-followup

Conversation

@c21

@c21 c21 commented Dec 7, 2021

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This is a follow up of #34702 (comment) , where we can replace object hash aggregate with sort aggregate as well. This PR is to handle object hash aggregate.

Why are the changes needed?

Increase coverage of rule by handling object hash aggregate as well.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Modified unit test in ReplaceHashWithSortAggSuite.scala to cover object hash aggregate (by using aggregate expression COLLECT_LIST).

@github-actions github-actions Bot added the SQL label Dec 7, 2021
@c21

c21 commented Dec 7, 2021

Copy link
Copy Markdown
Contributor Author

@cloud-fan could you help take a look when you have time? Thanks.

@SparkQA

SparkQA commented Dec 7, 2021

Copy link
Copy Markdown

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50443/

@SparkQA

SparkQA commented Dec 7, 2021

Copy link
Copy Markdown

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50443/

@cloud-fan

Copy link
Copy Markdown
Contributor

thanks, merging to master!

@cloud-fan cloud-fan closed this in 41a940f Dec 7, 2021
@c21

c21 commented Dec 7, 2021

Copy link
Copy Markdown
Contributor Author

Thank you @cloud-fan for review!

@c21 c21 deleted the agg-rule-followup branch December 7, 2021 04:48
@SparkQA

SparkQA commented Dec 7, 2021

Copy link
Copy Markdown

Test build #145968 has finished for PR 34824 at commit 0046770.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants