Skip to content

[SPARK-10038] [SQL] fix bug in generated unsafe projection when there is binary in ArrayData#8250

Closed
davies wants to merge 2 commits into
apache:masterfrom
davies:array_binary
Closed

[SPARK-10038] [SQL] fix bug in generated unsafe projection when there is binary in ArrayData#8250
davies wants to merge 2 commits into
apache:masterfrom
davies:array_binary

Conversation

@davies

@davies davies commented Aug 17, 2015

Copy link
Copy Markdown
Contributor

The type for array of array in Java is slightly different than array of others.

cc @cloud-fan

@JoshRosen

Copy link
Copy Markdown
Contributor

Jenkins, retest this please.

@SparkQA

SparkQA commented Aug 17, 2015

Copy link
Copy Markdown

Test build #41054 has finished for PR 8250 at commit 30e9182.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA

SparkQA commented Aug 18, 2015

Copy link
Copy Markdown

Test build #1636 has finished for PR 8250 at commit 30e9182.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • case class FilterNode(condition: Expression, child: LocalNode) extends UnaryLocalNode
    • abstract class LocalNode extends TreeNode[LocalNode]
    • abstract class LeafLocalNode extends LocalNode
    • abstract class UnaryLocalNode extends LocalNode
    • case class ProjectNode(projectList: Seq[NamedExpression], child: LocalNode) extends UnaryLocalNode
    • case class SeqScanNode(output: Seq[Attribute], data: Seq[InternalRow]) extends LeafLocalNode
    • public final class UTF8String implements Comparable<UTF8String>, Externalizable

@SparkQA

SparkQA commented Aug 18, 2015

Copy link
Copy Markdown

Test build #41058 has finished for PR 8250 at commit 31141d1.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan

Copy link
Copy Markdown
Contributor

Ah good catch! The change LGTM.

Not related to this PR, but as we have more and more hack(equals, hashCode, etc.) for binary type, should we create a wrapper class for it?

@rxin

rxin commented Aug 18, 2015

Copy link
Copy Markdown
Contributor

I've merged this. Thanks!

asfgit pushed a commit that referenced this pull request Aug 18, 2015
… is binary in ArrayData

The type for array of array in Java is slightly different than array of others.

cc cloud-fan

Author: Davies Liu <davies@databricks.com>

Closes #8250 from davies/array_binary.

(cherry picked from commit 5af3838)
Signed-off-by: Reynold Xin <rxin@databricks.com>
@asfgit asfgit closed this in 5af3838 Aug 18, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants