[SPARK-11889] [SQL] Fix type inference for GroupedDataset.agg in REPL#9870
[SPARK-11889] [SQL] Fix type inference for GroupedDataset.agg in REPL#9870marmbrus wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
@dragos any idea why it fails to infer the type only in the REPL?
There was a problem hiding this comment.
I can't reproduce the difference. It won't infer it in a standalone program either.
As I mentioned in our conversation, it's a chicken and egg problem: type inference is guided by the expected type, but if the method is overloaded, the expected type is not known. And the argument type is what guides overload resolution. It works in simple cases, when the overloads have different aritites, but with varargs that's no longer the case.
There was a problem hiding this comment.
It always works fine here though. Why is this different?
In this PR I delete a method that breaks type inference for aggregators (only in the REPL)
The error when this method is present is:
```
<console>:38: error: missing parameter type for expanded function ((x$2) => x$2._2)
ds.groupBy(_._1).agg(sum(_._2), sum(_._3)).collect()
```
Author: Michael Armbrust <michael@databricks.com>
Closes #9870 from marmbrus/dataset-repl-agg.
(cherry picked from commit 968acf3)
Signed-off-by: Michael Armbrust <michael@databricks.com>
|
Test build #46441 has finished for PR 9870 at commit
|
|
Ugh, sorry :( I'll fix this. |
In this PR I delete a method that breaks type inference for aggregators (only in the REPL)
The error when this method is present is: