Skip to content

[Feature](map) add map type to doris#15966

Merged
xy720 merged 12 commits into
apache:struct-typefrom
amorynan:feature-map
Feb 3, 2023
Merged

[Feature](map) add map type to doris#15966
xy720 merged 12 commits into
apache:struct-typefrom
amorynan:feature-map

Conversation

@amorynan

@amorynan amorynan commented Jan 16, 2023

Copy link
Copy Markdown
Contributor

Proposed changes

TO support doris complex type for map
Issue Number: close #xxx

Problem summary

Describe your changes.

Checklist(Required)

  1. Does it affect the original behavior:
    • Yes
    • No
    • I don't know
  2. Has unit tests been added:
    • Yes
    • No
    • No Need
  3. Has document been added or modified:
    • Yes
    • No
    • No Need
  4. Does it need to update dependencies:
    • Yes
    • No
  5. Are there any changes that cannot be rolled back:
    • Yes (If Yes, please explain WHY)
    • No

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@github-actions github-actions Bot added area/planner Issues or PRs related to the query planner area/sql/function Issues or PRs related to the SQL functions area/vectorization labels Jan 16, 2023

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp
Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp Outdated
Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp
Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp
Comment thread be/src/runtime/map_value.h Outdated
Comment thread be/src/vec/data_types/data_type_map.cpp Outdated
Comment on lines +124 to +125
if (!keys->equals(*rhs_map.keys))
return false;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
if (!keys->equals(*rhs_map.keys))
return false;
if (!keys->equals(*rhs_map.keys)) {
return false;
}

Comment thread be/src/vec/data_types/data_type_map.cpp Outdated
Comment thread be/src/vec/exprs/vmap_literal.cpp Outdated
Comment thread be/src/vec/exprs/vmap_literal.cpp Outdated
Comment thread be/src/vec/functions/array/function_array_element.h Outdated
@amorynan amorynan changed the base branch from master to struct-type January 16, 2023 10:05
@morningman

Copy link
Copy Markdown
Contributor

Please change the title and rebase the code

Comment thread be/src/olap/field.h Outdated
@amorynan amorynan changed the title feature-map [Feature](map)support complex struct for doris Jan 17, 2023

@xiaokang xiaokang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fe reviewed

Comment thread fe/fe-core/src/main/java/org/apache/doris/catalog/MapType.java
Comment thread fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java Outdated
Comment thread fe/fe-core/src/main/java/org/apache/doris/analysis/MapLiteral.java Outdated
Comment thread fe/fe-core/src/main/java/org/apache/doris/analysis/MapLiteral.java
Comment thread fe/fe-core/src/main/java/org/apache/doris/analysis/MapLiteral.java
Comment thread fe/fe-core/src/main/java/org/apache/doris/mysql/MysqlColType.java Outdated
Comment thread fe/fe-core/src/main/java/org/apache/doris/planner/SetOperationNode.java Outdated
Comment thread be/src/vec/data_types/data_type_map.cpp Outdated
Comment thread be/src/vec/data_types/data_type_map.cpp
Comment thread be/src/vec/data_types/data_type_map.cpp Outdated

@xiaokang xiaokang left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

be code reviewed

Comment thread be/src/vec/data_types/data_type_factory.hpp
Comment thread be/src/vec/data_types/data_type_factory.cpp Outdated
Comment thread be/src/vec/columns/column_map.cpp Outdated
Comment thread be/src/vec/columns/column_map.cpp Outdated
Comment thread be/src/vec/columns/column_map.cpp
Comment thread be/src/vec/functions/array/function_array_element.h Outdated
Comment thread be/src/vec/functions/array/function_array_element.h Outdated
Comment thread be/src/vec/functions/array/function_array_element.h Outdated
Comment thread be/src/vec/functions/array/function_array_element.h Outdated
Comment thread be/src/vec/sink/vmysql_result_writer.cpp
Comment thread be/src/olap/rowset/segment_v2/column_reader.cpp Outdated
Comment thread be/src/olap/rowset/segment_v2/column_reader.h Outdated
Comment thread be/src/olap/rowset/segment_v2/column_reader.h
Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp Outdated
Comment thread be/src/olap/types.h Outdated
Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp Outdated
Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp Outdated
Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp Outdated
Comment thread be/src/olap/rowset/segment_v2/column_writer.cpp Outdated
Comment thread be/src/vec/columns/column_map.h
Comment thread be/src/vec/columns/column_map.h
Comment thread be/src/vec/core/field.h
Comment thread fe/fe-core/src/main/java/org/apache/doris/analysis/CastExpr.java
Comment thread fe/fe-core/src/main/java/org/apache/doris/analysis/MapLiteral.java Outdated
Comment thread fe/fe-core/src/main/java/org/apache/doris/analysis/MapLiteral.java Outdated
Comment thread fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java Outdated
Comment thread fe/fe-core/src/main/java/org/apache/doris/catalog/MapType.java
Comment thread fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
@amorynan amorynan requested review from xiaokang and xy720 and removed request for xiaokang January 29, 2023 07:38

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/olap/rowset/segment_v2/column_reader.cpp Outdated
Comment thread be/src/olap/rowset/segment_v2/column_reader.cpp Outdated
Comment thread be/src/runtime/map_value.h Outdated
Comment thread be/src/runtime/map_value.h Outdated
Comment thread be/src/udf/udf.h Outdated
Comment thread be/src/vec/data_types/data_type_factory.cpp
Comment thread be/src/vec/data_types/data_type_factory.cpp
Comment thread be/src/vec/data_types/data_type_factory.cpp
Comment thread be/src/vec/data_types/data_type_factory.cpp
Comment thread be/src/vec/data_types/data_type_factory.cpp

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/runtime/types.cpp Outdated
Comment thread be/src/runtime/types.cpp Outdated
Comment thread be/src/vec/data_types/data_type_factory.cpp Outdated
Comment thread be/src/vec/data_types/data_type_factory.cpp Outdated
Comment thread be/src/vec/data_types/data_type_factory.cpp Outdated
Comment thread be/src/vec/sink/vmysql_result_writer.cpp Outdated
Comment thread be/src/vec/sink/vmysql_result_writer.cpp Outdated
Comment thread be/src/vec/sink/vmysql_result_writer.cpp Outdated
Comment thread be/src/olap/rowset/segment_v2/column_reader.cpp Outdated
Comment thread be/src/olap/rowset/segment_v2/column_reader.cpp Outdated
Comment thread be/src/olap/types.h Outdated
Comment thread be/src/runtime/types.cpp Outdated
Comment thread be/src/vec/olap/olap_data_convertor.cpp Outdated
Comment thread be/src/vec/sink/vmysql_result_writer.cpp Outdated
Comment thread fe/fe-core/src/main/java/org/apache/doris/analysis/CastExpr.java
@amorynan amorynan requested review from xiaokang and removed request for xiaokang February 1, 2023 10:48
@amorynan amorynan requested a review from xiaokang February 1, 2023 10:49

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 27. Check the log or trigger a new build to see more.

Comment thread be/src/runtime/types.cpp
Comment thread be/src/runtime/types.cpp
Comment thread be/src/runtime/types.cpp
Comment thread be/src/runtime/types.cpp
create_data_type(col_desc.children[1], col_desc.contains_nulls[1]));
break;
}
case INVALID_TYPE:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: 'case' statement not in switch statement [clang-diagnostic-error]

case INVALID_TYPE:
^

for (int i = 0; i < _children.size(); ++i) {
if (!_children[i]->is_constant()) {
return false;
std::string VExpr::debug_string(const std::vector<VExprContext*>& ctxs) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function definition is not allowed here [clang-diagnostic-error]

    std::string VExpr::debug_string(const std::vector<VExprContext*>& ctxs) {
                                                                            ^


return true;
}
bool VExpr::is_constant() const {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function definition is not allowed here [clang-diagnostic-error]

    bool VExpr::is_constant() const {
                                    ^

}

if (_constant_col != nullptr) {
Status VExpr::get_const_col(VExprContext * context, ColumnPtrWrapper * *output) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function definition is not allowed here [clang-diagnostic-error]

    Status VExpr::get_const_col(VExprContext * context, ColumnPtrWrapper * *output) {
                                                                                    ^

Comment thread be/src/vec/exprs/vexpr.cpp
Comment thread be/src/vec/exprs/vexpr.cpp
return Status::OK();
}

Status VExpr::create_tree_from_thrift(doris::ObjectPool* pool,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why modify here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I use clang-format.sh , it make these change...

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/vec/data_types/data_type_factory.cpp
Comment thread be/src/vec/data_types/data_type_factory.cpp

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

Comment thread be/src/vec/data_types/data_type_factory.cpp
Comment thread be/src/vec/data_types/data_type_factory.cpp
@amorynan amorynan changed the title [Feature](map)support complex struct for doris [Feature](map) add map type to doris Feb 2, 2023
@github-actions

github-actions Bot commented Feb 2, 2023

Copy link
Copy Markdown
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@amorynan amorynan requested review from xy720 and removed request for xiaokang and xy720 February 2, 2023 14:50

@xy720 xy720 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Feb 3, 2023
@github-actions

github-actions Bot commented Feb 3, 2023

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actions Bot commented Feb 3, 2023

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@amorynan amorynan requested a review from xiaokang February 3, 2023 04:04
@xy720 xy720 merged commit af9234b into apache:struct-type Feb 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. area/planner Issues or PRs related to the query planner area/sql/function Issues or PRs related to the SQL functions area/vectorization reviewed

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

5 participants