Skip to content

[SPARK-9549][SQL] fix bugs in expressions#7882

Closed
yjshen wants to merge 3 commits into
apache:masterfrom
yjshen:minor_bug_fix
Closed

[SPARK-9549][SQL] fix bugs in expressions#7882
yjshen wants to merge 3 commits into
apache:masterfrom
yjshen:minor_bug_fix

Conversation

@yjshen

@yjshen yjshen commented Aug 3, 2015

Copy link
Copy Markdown
Member

JIRA: https://issues.apache.org/jira/browse/SPARK-9549

This PR fix the following bugs:

  1. UnaryMinus's codegen version would fail to compile when the input is Long.MinValue
  2. BinaryComparison would fail to compile in codegen mode when comparing Boolean types.
  3. AddMonth would fail if passed a huge negative month, which would lead accessing negative index of monthDays array.
  4. Nanvl with different type operands.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add one for int/short/byte type too

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this throw a bad exception if the input is some other type that is not Double/Float?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have this in Nanvl:

override def inputTypes: Seq[AbstractDataType] =
    Seq(TypeCollection(DoubleType, FloatType), TypeCollection(DoubleType, FloatType))

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you double check that FunctionArgumentConversion is not called before the implicit cast rule?

@SparkQA

SparkQA commented Aug 3, 2015

Copy link
Copy Markdown

Test build #39500 has finished for PR 7882 at commit 4fa5de0.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Aug 3, 2015

Copy link
Copy Markdown

Test build #39505 has finished for PR 7882 at commit 3dee204.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin

rxin commented Aug 3, 2015

Copy link
Copy Markdown
Contributor

LGTM

@SparkQA

SparkQA commented Aug 3, 2015

Copy link
Copy Markdown

Test build #39514 has finished for PR 7882 at commit 41bbd2c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin

rxin commented Aug 3, 2015

Copy link
Copy Markdown
Contributor

Thanks - I've merged this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The above two statements can be replaced with:
val (currentYear, currentMonthInYear) = nonNegativeMonth /% 12

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is that possible?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/% is not defined for Int.

I read the notion in a Scala book which I have returned. I will read more once I have that book back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants