Skip to content

ARROW-7960: [C++] Add support fo reading additional types#8376

Closed
emkornfield wants to merge 2 commits into
apache:masterfrom
emkornfield:read_most_types
Closed

ARROW-7960: [C++] Add support fo reading additional types#8376
emkornfield wants to merge 2 commits into
apache:masterfrom
emkornfield:read_most_types

Conversation

@emkornfield

@emkornfield emkornfield commented Oct 7, 2020

Copy link
Copy Markdown
Contributor

New types supported:

  • Fixed Size list (will throw an incorrect error if nulls). Null slot count is best implemented after we recursively apply metadata.
  • LargeList
  • Maps

Still missing: LargeString and LargeBytes

Unimplemented functionality:

  • Removing duplicate maps on read.
  • Single column maps are converted to List of struct.

Other:
Fixed two bugs for FixedSizeLists:

  1. Def levels genereated were incorrect (they only made sense for
    two level lists).
  2. Slices where not handled appropriately

@emkornfield emkornfield requested a review from pitrou October 7, 2020 07:56
@github-actions

github-actions Bot commented Oct 7, 2020

Copy link
Copy Markdown

@pitrou pitrou changed the title ARROW-7960: Add support fo reading additional types ARROW-7960: [C++] Add support fo reading additional types Oct 7, 2020
New types supported:
- Fixed Size list (will throw an incorrect error if nulls
  are present, but this is best fixed after we recursively support
  applying types).
- LargeList
- Maps

Still missing: LargeString and LargeBytes

Unimplemented functionality:
  - Deduping maps on read.

Single column maps are converted to List of struct.

Other Fixed two bugs for FixedSizeLists:
1.  Def levels genereated were incorrect (they only made sense for
    two level lists).
2.  Slices where not handled appropriately
@pitrou

pitrou commented Oct 7, 2020

Copy link
Copy Markdown
Member

Rebased, fixed conflicts.

@pitrou pitrou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Thank you very much @emkornfield .

@pitrou

pitrou commented Oct 7, 2020

Copy link
Copy Markdown
Member

@pitrou

pitrou commented Oct 7, 2020

Copy link
Copy Markdown
Member

CI failures are unrelated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants