Skip to content

[pick](expr) pick expr modify to branch4.0#59241

Merged
yiguolei merged 6 commits into
apache:branch-4.0from
Mryange:pick-expr-to-4.0
Dec 23, 2025
Merged

[pick](expr) pick expr modify to branch4.0#59241
yiguolei merged 6 commits into
apache:branch-4.0from
Mryange:pick-expr-to-4.0

Conversation

@Mryange

@Mryange Mryange commented Dec 22, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

pick

#56905

#58125

#58692

#58578

#58710

#58841

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Mryange
[refine](condition) Change conditional functions to native expr (apac…
912e575 
…he#58125)

Previously our conditional functions used the function framework.

For example: if(cond, exprA, exprB). Before executing if, cond, exprA,
and exprB were already evaluated.

By switching to expr, we can choose to evaluate either exprA or exprB
based on the actual value of cond.

Switching to expr also paves the way to support short-circuit evaluation
in the future.
@Mryange
[fix](core) Fix count_true_with_notnull failing to correctly handle C…
4fcf889 
…onst (Nullable) cases. (apache#58692)

apache#58125

```
F20251203 19:47:55.046221 2067836 status.h:467] Bad cast from type:doris::vectorized::ColumnConst* to doris::vectorized::ColumnVector<(doris::PrimitiveType)2> const*
*** Check failure stack trace: ***
    @     0x55d7d5bc798f  google::LogMessage::SendToLog()
    @     0x55d7d5bbdfa0  google::LogMessage::Flush()
    @     0x55d7d5bc1699  google::LogMessageFatal::~LogMessageFatal()
    @     0x55d7a87a211b  doris::Status::FatalError<>()
    @     0x55d7c8732ed6  _ZZ11assert_castIPKN5doris10vectorized12ColumnVectorILNS0_13PrimitiveTypeE2EEEL18TypeCheckOnRelease1EPKNS1_7IColumnEET_OT1_ENKUlOSB_E_clISA_EES6_SE_
    @     0x55d7c87322e4  assert_cast<>()
    @     0x55d7c96b4334  doris::vectorized::VConditionExpr::count_true_with_notnull()
    @     0x55d7c96c2edb  doris::vectorized::VectorizedIfExpr::execute_column()
    @     0x55d7b02f867d  doris::vectorized::VExpr::execute()
    @     0x55d7c94e1a4b  doris::vectorized::VExpr::get_const_col()
    @     0x55d7c94e0cfe  doris::vectorized::VExpr::open()
    @     0x55d7c96b246e  doris::vectorized::VConditionExpr::open()
    @     0x55d7c9881761  doris::vectorized::VExprContext::open()
    @     0x55d7c94eaeac  doris::vectorized::VExpr::open()
```
@Mryange
[refine](expr) Expr execution does not depend on the block's rows. (a…
1b1b30c 
…pache#58578)

Previously, when executing an expr we relied on block.rows() to
determine the number of rows to return. That required guaranteeing the
block was not empty and that the first column (since block.rows() is
implemented from it) had the final desired row count. Now we pass a
count parameter to indicate the size each expr should return for its
Column.

before
```C++
vectorized::Block tmp_block;
        tmp_block.insert({vectorized::ColumnUInt8::create(1),
                          std::make_shared<vectorized::DataTypeUInt8>(), ""});

RETURN_IF_ERROR(_const_expr_lists[_const_expr_list_idx][i]->execute(&tmp_block,
                                                                                &result_list[i]));
```
now
```C++
RETURN_IF_ERROR(_const_expr_lists[_const_expr_list_idx][i]->execute_const_expr(
                    tmp_block_columns[i]));

Status VExprContext::execute_const_expr(ColumnWithTypeAndName& result) {
    Status st;
    RETURN_IF_CATCH_EXCEPTION({ st = _root->execute_column(this, nullptr, 1, result.column); });
```
@Mryange
[chore](exec) BE constant folding no longer needs to construct a temp…
15e7272 
…orary block (apache#58710)

Also, this PR fixes a bug: previously BE constant folding did not pass
whether casts should be performed in strict mode.

```
W20251205 13:32:09.197405 512888 internal_service.cpp:1608] exec fold constant expr failed, errmsg=[INTERNAL_ERROR]ColumnWithTypeAndName check column type failed, column name: (CAST String(String) TO IPv4), type: IPv4,  column: Const(Nullable(IPV4)) , error: [INTERNAL_ERROR]Column type Const(Nullable(IPV4)) is not compatible with data type IPv4        0#  doris::Status doris::vectorized::IDataType::check_column_non_nested_type<doris::vectorized::ColumnVector<(doris::PrimitiveType)36> >(doris::vectorized::IColumn const&) const at /root/doris/be/src/common/status.h:0
        1#  doris::vectorized::DataTypeNumberBase<(doris::PrimitiveType)36>::check_column(doris::vectorized::IColumn const&) const at /root/doris/be/src/vec/data_types/data_type_number_base.cpp:239
        2#  doris::vectorized::ColumnWithTypeAndName::check_type_and_column_match() const at /root/doris/be/src/vec/core/column_with_type_and_name.cpp:0
        3#  doris::vectorized::VExprContext::execute(doris::vectorized::Block*, int*) at /root/doris/be/src/vec/exprs/vexpr_context.cpp:0
        4#  doris::FoldConstantExecutor::fold_constant_vexpr(doris::TFoldConstantParams const&, doris::PConstantExprResult*) at /root/doris/be/src/common/status.h:0
        5#  std::_Function_handler<void (), doris::PInternalService::fold_constant_expr(google::protobuf::RpcController*, doris::PConstantExprRequest const*, doris::PConstantExprResult*, google::protobuf::Closure*)::$_0>::_M_invoke(std::_Any_data const&) at /root/doris/be/src/common/status.h:0
```
@Mryange
[chore](code)Remove the unused member variable _getting_const_col and…
1824bea 
… get_const_col no longer constructs a block. (apache#58841)

Currently this variable is only used in DCHECK. Whether to apply the
const optimization (is_const_and_have_executed) does not depend on this
variable.

```
    bool is_const_and_have_executed() const {
        return (is_constant() && (_constant_col != nullptr));
    }
```

We can determine whether we're currently executing a constant expression
by checking if block is nullptr.

get_const_col no longer constructs a block

```
    int result = -1;
    Block block;
    // If block is empty, some functions will produce no result. So we insert a column with
    // single value here.
    block.insert({ColumnUInt8::create(1), std::make_shared<DataTypeUInt8>(), ""});

    RETURN_IF_ERROR(execute(context, &block, &result));
```

now

```
    ColumnPtr result;
    RETURN_IF_ERROR(execute_column(context, nullptr, 1, result));
    _constant_col = std::make_shared<ColumnPtrWrapper>(result);
```
@Mryange Mryange requested a review from yiguolei as a code owner December 22, 2025 06:17
@Thearas

Thearas commented Dec 22, 2025

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@Mryange Mryange changed the title [pick](expr) pick expr modify [pick](expr) pick expr modify to branch4.0 Dec 22, 2025
@Mryange

Mryange commented Dec 22, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Dec 22, 2025

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 7.79% (58/745) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.07% (18474/34810)
Line Coverage 38.75% (170214/439316)
Region Coverage 33.57% (131680/392260)
Branch Coverage 34.43% (56807/164995)

@github-actions github-actions Bot added the approved Indicates a PR has been approved by one committer. label Dec 23, 2025
@github-actions

github-actions Bot commented Dec 23, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions

github-actions Bot commented Dec 23, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 82f1099 into apache:branch-4.0 Dec 23, 2025
23 of 27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Mryange @Thearas @doris-robot @yiguolei