Skip to content

[WIP] remove expensive api from InternalRow#6869

Closed
davies wants to merge 1 commit into
apache:masterfrom
davies:internalrow2
Closed

[WIP] remove expensive api from InternalRow#6869
davies wants to merge 1 commit into
apache:masterfrom
davies:internalrow2

Conversation

@davies

@davies davies commented Jun 18, 2015

Copy link
Copy Markdown
Contributor

This is a follow up PR for #6792, to use narrow down the APIs in InternalRow (remove those expensive ones).

@davies

davies commented Jun 18, 2015

Copy link
Copy Markdown
Contributor Author

cc @liancheng @rxin @JoshRosen

@cloud-fan

Copy link
Copy Markdown
Contributor

hi @davies , I met a bug at #6825 that we didn't handle BinaryType(byte[]) correctly at equals and hashCode. We should use java.util.Arrays.equals and java.util.Arrays.hashCode. Could you fix it here?

@SparkQA

SparkQA commented Jun 18, 2015

Copy link
Copy Markdown

Test build #35092 has finished for PR 6869 at commit 444382e.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait Row extends InternalRow
    • abstract class InternalRow extends Serializable
    • final class SpecificMutableRow(val values: Array[MutableValue]) extends Row with MutableRow
    • class GenericRow(protected[sql] val values: Array[Any]) extends Row
    • abstract class InternalOutputWriter extends OutputWriter

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need size? maybe just keep one of length or size.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree. It's internal and we don't need to follow scala convention to provide size.

@rxin

rxin commented Jun 18, 2015

Copy link
Copy Markdown
Contributor

@davies can you update the pr description to include what you actually removed?

@davies

davies commented Jun 18, 2015

Copy link
Copy Markdown
Contributor Author

@cloud-fan it will be fixed by #6876

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it's internal row, should we just return UTF8String?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can only define getUTF8String at InternalRow and define getString at user-facing Row?

@davies

davies commented Jun 24, 2015

Copy link
Copy Markdown
Contributor Author

Close this, in favor of #7003

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants