[SPARK-20783][SQL][Follow-up] Create ColumnVector to abstract existing compressed column#19508
[SPARK-20783][SQL][Follow-up] Create ColumnVector to abstract existing compressed column#19508viirya wants to merge 2 commits into
Conversation
|
cc @kiszk Please check if this change is proper. |
|
Test build #82804 has finished for PR 19508 at commit
|
|
cc @cloud-fan too |
|
I will check this on Wed. |
|
|
||
| protected def underlyingBuffer = buffer | ||
|
|
||
| def getByteBuffer: ByteBuffer = |
| columnType.dataType match { | ||
| case _: IntegerType => | ||
| val dictionaryIds = columnVector.reserveDictionaryIds(capacity) | ||
| val intDictionary = dictionary.map(_.asInstanceOf[Int]) |
There was a problem hiding this comment.
@viirya Could you please see this comment to avoid unboxing? IIUC, this code still causes unboxing.
What do you think?
There was a problem hiding this comment.
I didn't notice that comment. Let me check it then. If we can't avoid, I will revert this and only remove the unused method. Thanks.
There was a problem hiding this comment.
Thanks @kiszk. I checked the comment and code and reverted this change.
|
Test build #82897 has finished for PR 19508 at commit
|
|
ping @cloud-fan for final check. Thanks. |
|
LGTM |
|
ping @cloud-fan Please take a quick look. Thanks. |
|
re-ping @cloud-fan This is a simple follow-up, please check it. Thanks. |
|
thanks, merging to master! |
4 participants
What changes were proposed in this pull request?
Removed one unused method.
How was this patch tested?
Existing tests.