Skip to content

SPARK-11716 - UDFRegistration just drops the input type when re-creat…#9739

Closed
jbonofre wants to merge 2 commits into
apache:masterfrom
jbonofre:SPARK-11716
Closed

SPARK-11716 - UDFRegistration just drops the input type when re-creat…#9739
jbonofre wants to merge 2 commits into
apache:masterfrom
jbonofre:SPARK-11716

Conversation

@jbonofre

@jbonofre jbonofre commented Nov 16, 2015

Copy link
Copy Markdown
Member

…ing the UserDefinedFunction

@const-ae

const-ae commented Nov 17, 2015

Copy link
Copy Markdown

Hi, I guess you should also update the script in the comments that generates the file.

@jbonofre

jbonofre commented Nov 17, 2015

Copy link
Copy Markdown
Member Author

Good point, let me update the PR accordingly. Thanks.

@jbonofre

jbonofre commented Nov 17, 2015

Copy link
Copy Markdown
Member Author

PR rebased and updated according to @Artjom-Metro comment.

@jbonofre

jbonofre commented Nov 18, 2015

Copy link
Copy Markdown
Member Author

Rebase

@yhuai

yhuai commented Nov 18, 2015

Copy link
Copy Markdown
Contributor

Can you change the title format to [SPARK-11716] [SQL] ...?

@yhuai

yhuai commented Nov 18, 2015

Copy link
Copy Markdown
Contributor

ok to test.

@yhuai

yhuai commented Nov 18, 2015

Copy link
Copy Markdown
Contributor

Can you add a test?

@yhuai

yhuai commented Nov 18, 2015

Copy link
Copy Markdown
Contributor

So, the problem is that when we use apply method on this UDF, we will not be able to pass in the type and thus, we fail to do type casting, right? It is not the case for using expr because the udf we get from the function registry has the input type. It will be great to update the description to describe the problem.

@SparkQA

SparkQA commented Nov 19, 2015

Copy link
Copy Markdown

Test build #46257 has finished for PR 9739 at commit 3ebf487.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@jbonofre

jbonofre commented Nov 19, 2015

Copy link
Copy Markdown
Member Author

@yhuai Thanks for your feedbacks. Let me update the PR accordingly.

@yhuai yhuai mentioned this pull request Nov 20, 2015
Closed
asfgit pushed a commit that referenced this pull request Nov 20, 2015
@jbonofre @marmbrus
[SPARK-11716][SQL] UDFRegistration just drops the input type when re-…
b9b0e17 
…creating the UserDefinedFunction

https://issues.apache.org/jira/browse/SPARK-11716

This is one is #9739 and a regression test. When commit it, please make sure the author is jbonofre.

You can find the original PR at #9739

closes #9739

Author: Jean-Baptiste Onofré <jbonofre@apache.org>
Author: Yin Huai <yhuai@databricks.com>

Closes #9868 from yhuai/SPARK-11716.

(cherry picked from commit 03ba56d)
Signed-off-by: Michael Armbrust <michael@databricks.com>
@asfgit asfgit closed this in 03ba56d Nov 20, 2015
@jbonofre

jbonofre commented Nov 21, 2015

Copy link
Copy Markdown
Member Author

Awesome, thanks a lot @yhuai !

@yhuai

yhuai commented Nov 21, 2015

Copy link
Copy Markdown
Contributor

no problem! Thank you for your fix!

kiszk pushed a commit to kiszk/spark-gpu that referenced this pull request Dec 26, 2015
@jbonofre @kiszk
[SPARK-11716][SQL] UDFRegistration just drops the input type when re-…
d4760d5 
…creating the UserDefinedFunction

https://issues.apache.org/jira/browse/SPARK-11716

This is one is #9739 and a regression test. When commit it, please make sure the author is jbonofre.

You can find the original PR at apache/spark#9739

closes #9739

Author: Jean-Baptiste Onofré <jbonofre@apache.org>
Author: Yin Huai <yhuai@databricks.com>

Closes #9868 from yhuai/SPARK-11716.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jbonofre @const-ae @yhuai @SparkQA