[fix](arrow) Rollback Arrow version 19.0.1 -> 17.0.0, Fix MacOS compile error and decimal type error when convert to Parquet#51217
Merged
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
TPC-H: Total hot run time: 33635 ms |
TPC-DS: Total hot run time: 192782 ms |
ClickBench: Total hot run time: 29.05 s |
xinyiZzz
changed the title
xinyiZzz
changed the title
xinyiZzz
changed the title
Contributor
Author
|
run buildall |
xinyiZzz
force-pushed
the
20250525_fix_arrow_version
branch
from
d35176f to
e64c5f2
Compare
Contributor
Author
|
run buildall |
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
Contributor
|
PR approved by anyone and no changes requested. |
TPC-H: Total hot run time: 33713 ms |
TPC-DS: Total hot run time: 192612 ms |
ClickBench: Total hot run time: 28.82 s |
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
zy-kkk
previously approved these changes
May 26, 2025
Contributor
Author
|
run buildall |
github-actions
Bot
removed
the
approved
yiguolei
approved these changes
May 26, 2025
github-actions
Bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
TPC-H: Total hot run time: 33850 ms |
TPC-DS: Total hot run time: 186978 ms |
ClickBench: Total hot run time: 29.48 s |
zy-kkk
approved these changes
May 26, 2025
CalvinKirs
approved these changes
May 26, 2025
xinyiZzz
added a commit
to xinyiZzz/incubator-doris
that referenced
this pull request
May 27, 2025
…le error and decimal type error when convert to Parquet (apache#51217) Introduced: apache#50939 In Arrow 19.0.1, 1. some scenarios will force the use of `arrow::smallest_decimal` instead of `arrow::decimal`, but when `arrow::smallest_decimal` is converted to Parquet, an error will occur: `NotImplemented: Unhandled type for Arrow to Parquet schema conversion: decimal32(5, 2)` 2. when compile Doris BE on MacOS M1, an error `arithmetic on a pointer to an incomplete type 'arrow::flight::FlightEndpoint'` will occur. This is because there is a circular dependency in `arrow/flight/types.h`, `std::vector<FlightEndpoint>` is used in `476: struct Data`, but `809: struct FlightEndpoint` is defined later.
16 tasks
xinyiZzz
added a commit
to xinyiZzz/incubator-doris
that referenced
this pull request
May 27, 2025
…le error and decimal type error when convert to Parquet (apache#51217) Introduced: apache#50939 In Arrow 19.0.1, 1. some scenarios will force the use of `arrow::smallest_decimal` instead of `arrow::decimal`, but when `arrow::smallest_decimal` is converted to Parquet, an error will occur: `NotImplemented: Unhandled type for Arrow to Parquet schema conversion: decimal32(5, 2)` 2. when compile Doris BE on MacOS M1, an error `arithmetic on a pointer to an incomplete type 'arrow::flight::FlightEndpoint'` will occur. This is because there is a circular dependency in `arrow/flight/types.h`, `std::vector<FlightEndpoint>` is used in `476: struct Data`, but `809: struct FlightEndpoint` is defined later.
yiguolei
pushed a commit
that referenced
this pull request
May 27, 2025
… MacOS compile error and decimal type error when convert to Parquet (#51291) pick #51217 ### What problem does this PR solve? Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
koarz
pushed a commit
to koarz/doris
that referenced
this pull request
Jun 4, 2025
…le error and decimal type error when convert to Parquet (apache#51217) ### What problem does this PR solve? Introduced: apache#50939 In Arrow 19.0.1, 1. some scenarios will force the use of `arrow::smallest_decimal` instead of `arrow::decimal`, but when `arrow::smallest_decimal` is converted to Parquet, an error will occur: `NotImplemented: Unhandled type for Arrow to Parquet schema conversion: decimal32(5, 2)` 2. when compile Doris BE on MacOS M1, an error `arithmetic on a pointer to an incomplete type 'arrow::flight::FlightEndpoint'` will occur. This is because there is a circular dependency in `arrow/flight/types.h`, `std::vector<FlightEndpoint>` is used in `476: struct Data`, but `809: struct FlightEndpoint` is defined later.
morrySnow
added
not-merge/3.0
not-merge/3.1
and removed
dev/3.0.x
dev/3.0.x-conflict
dev/3.1.x
dev/3.1.x-conflict
labels
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Introduced: #50939
In Arrow 19.0.1,
some scenarios will force the use of
arrow::smallest_decimalinstead ofarrow::decimal, but whenarrow::smallest_decimalis converted to Parquet, an error will occur:NotImplemented: Unhandled type for Arrow to Parquet schema conversion: decimal32(5, 2)when compile Doris BE on MacOS M1, an error
arithmetic on a pointer to an incomplete type 'arrow::flight::FlightEndpoint'will occur. This is because there is a circular dependency inarrow/flight/types.h,std::vector<FlightEndpoint>is used in476: struct Data, but809: struct FlightEndpointis defined later.Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)