Skip to content

[SPARK-13087][SQL] Fix group by function for sort based aggregation#11011

Closed
marmbrus wants to merge 1 commit into
apache:branch-1.6from
marmbrus:groupByFunction
Closed

[SPARK-13087][SQL] Fix group by function for sort based aggregation#11011
marmbrus wants to merge 1 commit into
apache:branch-1.6from
marmbrus:groupByFunction

Conversation

@marmbrus

@marmbrus marmbrus commented Feb 2, 2016

Copy link
Copy Markdown
Contributor

It is not valid to call toAttribute on a NamedExpression unless we know for sure that the child produced that NamedExpression. The current code worked fine when the grouping expressions were simple, but when they were a derived value this blew up at execution time.

@marmbrus

marmbrus commented Feb 2, 2016

Copy link
Copy Markdown
Contributor Author

/cc @cloud-fan

@marmbrus

marmbrus commented Feb 2, 2016

Copy link
Copy Markdown
Contributor Author

I'm investigating if this is a problem in master and will open another PR if needed.

@marmbrus

marmbrus commented Feb 2, 2016

Copy link
Copy Markdown
Contributor Author

Opened #11013 for master

@SparkQA

SparkQA commented Feb 2, 2016

Copy link
Copy Markdown

Test build #50519 has finished for PR 11011 at commit e605a09.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan

Copy link
Copy Markdown
Contributor

good catch! I think it's a mistake when we write planAggregateWithoutPartial, there is no intermediate layer and we can't assume the child will produce that NamedExpression.

LGTM

@yhuai

yhuai commented Feb 2, 2016

Copy link
Copy Markdown
Contributor

Merging to branch 1.6.

asfgit pushed a commit that referenced this pull request Feb 2, 2016
It is not valid to call `toAttribute` on a `NamedExpression` unless we know for sure that the child produced that `NamedExpression`.  The current code worked fine when the grouping expressions were simple, but when they were a derived value this blew up at execution time.

Author: Michael Armbrust <michael@databricks.com>

Closes #11011 from marmbrus/groupByFunction.
@yhuai

yhuai commented Feb 2, 2016

Copy link
Copy Markdown
Contributor

Merged!

@marmbrus marmbrus closed this Feb 2, 2016
@marmbrus marmbrus deleted the groupByFunction branch March 8, 2016 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants