Skip to content

[SPARK-9927][SQL]Revert 8049 since it's pushing wrong filter down#8157

Closed
yjshen wants to merge 1 commit into
apache:masterfrom
yjshen:rever8049
Closed

[SPARK-9927][SQL]Revert 8049 since it's pushing wrong filter down#8157
yjshen wants to merge 1 commit into
apache:masterfrom
yjshen:rever8049

Conversation

@yjshen

@yjshen yjshen commented Aug 13, 2015

Copy link
Copy Markdown
Member

I made a mistake in #8049 by casting literal value to attribute's data type, which would cause simply truncate the literal value and push a wrong filter down.

JIRA: https://issues.apache.org/jira/browse/SPARK-9927

@liancheng

Copy link
Copy Markdown
Contributor

@yjshen Could you provide a snippet that reproduces the error you mentioned?

@yjshen

yjshen commented Aug 13, 2015

Copy link
Copy Markdown
Member Author

select * from inttypes where A > 2147483648, it would produce GreaterThan(Cast(A, LongType), Literal(2147483648L, LongType), in #8049 I just cast the long value to IntType, therefore the pushed down filter became A > -2147483648.

@SparkQA

SparkQA commented Aug 13, 2015

Copy link
Copy Markdown

Test build #40723 has finished for PR 8157 at commit db15678.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class QRDecomposition[QType, RType](Q: QType, R: RType)

@liancheng

Copy link
Copy Markdown
Contributor

@yjshen Thanks, I'm merging this to master and branch-1.5 and reopening SPARK-9182.

asfgit pushed a commit that referenced this pull request Aug 13, 2015
I made a mistake in #8049 by casting literal value to attribute's data type, which would cause simply truncate the literal value and push a wrong filter down.

JIRA: https://issues.apache.org/jira/browse/SPARK-9927

Author: Yijie Shen <henry.yijieshen@gmail.com>

Closes #8157 from yjshen/rever8049.

(cherry picked from commit d0b1891)
Signed-off-by: Cheng Lian <lian@databricks.com>
@asfgit asfgit closed this in d0b1891 Aug 13, 2015
@yjshen

yjshen commented Aug 13, 2015

Copy link
Copy Markdown
Member Author

@liancheng , I'm really sorry for my mistake.

@liancheng

Copy link
Copy Markdown
Contributor

Nah, I suggested the wrong fix at the first place anyway :)

CodingCat pushed a commit to CodingCat/spark that referenced this pull request Aug 17, 2015
I made a mistake in apache#8049 by casting literal value to attribute's data type, which would cause simply truncate the literal value and push a wrong filter down.

JIRA: https://issues.apache.org/jira/browse/SPARK-9927

Author: Yijie Shen <henry.yijieshen@gmail.com>

Closes apache#8157 from yjshen/rever8049.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants