Refine OR null masking for search function#3
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the ✨ Finishing touches🧪 Generate unit tests
Tip 👮 Agentic pre-merge checks are now available in preview!Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.
Please see the documentation for more information. Example: reviews:
pre_merge_checks:
custom_checks:
- name: "Undocumented Breaking Changes"
mode: "warning"
instructions: |
Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).Please share your feedback with us on this Discord post. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…armup` due to capture by reference (apache#56395) ### What problem does this PR solve? introduced in apache#54611 ``` (gdb) bt #0 __GI___pthread_sigmask (how=2, newmask=<optimized out>, oldmask=0x0) at ./nptl/pthread_sigmask.c:43 #1 0x00007f7aa6b5d71e in PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #2 0x00007f7aa6b5e206 in JVM_handle_linux_signal () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #3 <signal handler called> apache#4 0x000055e20050d443 in std::_Hashtable<doris::RowsetId, std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo>, std::allocator<std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo> >, std::__detail::_Select1st, std::equal_to<doris::RowsetId>, std::hash<doris::RowsetId>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_find_before_node (this=0x7f79e69acde0, __bkt=4186920012728959759, __k=..., __code=11200260987994981938) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/hashtable.h:2205 apache#5 std::_Hashtable<doris::RowsetId, std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo>, std::allocator<std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo> >, std::__detail::_Select1st, std::equal_to<doris::RowsetId>, std::hash<doris::RowsetId>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::_M_locate (this=this@entry=0x7f79e69acde0, __k=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/hashtable.h:2283 apache#6 0x000055e200505a8d in std::_Hashtable<doris::RowsetId, std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo>, std::allocator<std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo> >, std::__detail::_Select1st, std::equal_to<doris::RowsetId>, std::hash<doris::RowsetId>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >::find (this=0x7f79e69acde0, __k=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/hashtable.h:1929 apache#7 std::unordered_map<doris::RowsetId, doris::CloudTablet::RowsetWarmUpInfo, std::hash<doris::RowsetId>, std::equal_to<doris::RowsetId>, std::allocator<std::pair<doris::RowsetId const, doris::CloudTablet::RowsetWarmUpInfo> > >::contains (this=0x7f79e69acde0, __x=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/unordered_map.h:999 apache#8 doris::CloudTablet::complete_rowset_segment_warmup (this=<optimized out>, rowset_id=..., status=..., segment_num=1, inverted_idx_num=0) at /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_tablet.cpp:1692 apache#9 0x000055e2005338ba in doris::CloudWarmUpManager::handle_jobs()::$_0::operator()(doris::Status) const (this=0x7f79e8bebad0, st=...) at /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_warm_up_manager.cpp:243 apache#10 std::__invoke_impl<void, doris::CloudWarmUpManager::handle_jobs()::$_0&, doris::Status>(std::__invoke_other, doris::CloudWarmUpManager::handle_jobs()::$_0&, doris::Status&&) (__f=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#11 std::__invoke_r<void, doris::CloudWarmUpManager::handle_jobs()::$_0&, doris::Status>(doris::CloudWarmUpManager::handle_jobs()::$_0&, doris::Status&&) (__fn=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113 apache#12 std::_Function_handler<void (doris::Status), doris::CloudWarmUpManager::handle_jobs()::$_0>::_M_invoke(std::_Any_data const&, doris::Status&&) (__functor=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292 apache#13 0x000055e200533310 in std::function<void (doris::Status)>::operator()(doris::Status) const (this=0x7f795a3ff1d0, __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#14 doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0::operator()(doris::Status) const ( this=0x7f79f0710b60, st=...) at /home/zcp/repo_center/doris_master/doris/be/src/cloud/cloud_warm_up_manager.cpp:149 apache#15 std::__invoke_impl<void, doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0&, doris::Status>(std::__invoke_other, doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0&, doris::Status&&) ( __f=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#16 std::__invoke_r<void, doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0&, doris::Status>(doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0&, doris::Status&&) (__fn=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113 apache#17 std::_Function_handler<void (doris::Status), doris::CloudWarmUpManager::submit_download_tasks(std::filesystem::__cxx11::path, long, std::shared_ptr<doris::io::FileSystem>, long, std::shared_ptr<bthread::CountdownEvent>, bool, std::function<void (doris::Status)>)::$_0>::_M_invoke(std::_Any_data const&, doris::Status&&) (__functor=..., __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292 apache#18 0x000055e200469cb1 in std::function<void (doris::Status)>::operator()(doris::Status) const (this=0x7f796941cee0, __args=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#19 doris::io::FileCacheBlockDownloader::download_segment_file (this=<optimized out>, meta=...) at /home/zcp/repo_center/doris_master/doris/be/src/io/cache/block_file_cache_downloader.cpp:297 apache#20 0x000055e1fbfcbe85 in doris::ThreadPool::dispatch_thread (this=0x7f7995a1da00) at /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:614 apache#21 0x000055e1fbfc0fac in std::function<void ()>::operator()() const (this=0x7f796941cee0) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#22 doris::Thread::supervise_thread (arg=0x7f79f13de010) at /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:460 apache#23 0x00007f7aa5bf8ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#24 0x00007f7aa5c8a850 in __closefrom_fallback (from=1685031872, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#25 0x0000000000000000 in ?? () ``` Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
```
/home/zcp/repo_center/doris_master/doris/be/src/util/counts.h:127:85: runtime error: signed integer overflow: 147483648 - -2147483648 cannot be represented in type 'int'
#0 0x558c5a9d58cd in doris::Counts::terminate(double) /home/zcp/repo_center/doris_master/doris/be/src/util/counts.h:127:85
#1 0x558c5ab0ce99 in doris::vectorized::PercentileState<(doris::PrimitiveType)5>::insert_result_into(doris::vectorized::IColumn&) const /home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function_percentile.h:430:49
#2 0x558c5ab0812f in doris::vectorized::AggregateFunctionPercentileArray<(doris::PrimitiveType)5>::insert_result_into(char const*, doris::vectorized::IColumn&) const /home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function_percentile.h:557:59
#3 0x558c5ab086a8 in doris::vectorized::IAggregateFunctionHelper>::insert_result_into_vec(std::vector> const&, unsigned long, doris::vectorized::IColumn&, unsigned long) const /home/zcp/repo_center/doris_master/doris/be/src/vec/aggregate_functions/aggregate_function.h:393:22
apache#4 0x558c68af8445 in _ZZN5doris8pipeline13AggLocalState31_get_with_serialized_key_resultEPNS_12RuntimeStateEPNS_10vectorized5BlockEPbENK3$_1clINS4_15MethodKeysFixedI9PHHashMapIN4wide7integerILm256EjEEPc9HashCRC32ISE_EEEEEEvRT_ /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:287:67
apache#5 0x558c68af8445 in void std::__invoke_impl, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&>(std::__invoke_other, doris::vectorized::Overload&&, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14
apache#6 0x558c68af8445 in std::__invoke_result, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&>::type std::__invoke, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&>(doris::vectorized::Overload&&, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:98:14
apache#7 0x558c68af8445 in std::__detail::__variant::__gen_vtable_impl (*)(doris::vectorized::Overload&&, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&)>, std::integer_sequence>::__visit_invoke(doris::vectorized::Overload&&, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/variant:1055:11
apache#8 0x558c68a85790 in decltype(auto) std::__do_visit, doris::vectorized::Overload, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>(doris::vectorized::Overload&&, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/variant:1858:15
apache#9 0x558c68a85790 in std::invoke_result, std::__conditional>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>>::type>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>()))>::type>::type&, std::variant_alternative<0ul, std::remove_reference>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>()))>::type>::type&&>>::type std::visit, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&>(doris::vectorized::Overload&&, std::variant>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodOneNumber>>, doris::vectorized::MethodStringNoCache>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber, PHHashMap, char*, HashCRC32>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodOneNumber>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn, doris::vectorized::DataWithNullKey, char*, HashCRC32>>>>>, doris::vectorized::MethodSingleNullableColumn>>>>, doris::vectorized::MethodKeysFixed>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed, char*, HashCRC32>>>, doris::vectorized::MethodKeysFixed>>>&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/variant:1954:13
apache#10 0x558c68a85790 in doris::pipeline::AggLocalState::_get_with_serialized_key_result(doris::RuntimeState*, doris::vectorized::Block*, bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:251:5
apache#11 0x558c68b3e3e7 in doris::pipeline::AggLocalState::init(doris::RuntimeState*, doris::pipeline::LocalStateInfo&)::$_2::operator()(doris::RuntimeState*, doris::vectorized::Block*, bool*) const /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:75:24
apache#12 0x558c68b3e3e7 in doris::Status std::__invoke_impl(std::__invoke_other, doris::pipeline::AggLocalState::init(doris::RuntimeState*, doris::pipeline::LocalStateInfo&)::$_2&, doris::RuntimeState*&&, doris::vectorized::Block*&&, bool*&&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14
apache#13 0x558c68b3e3e7 in std::enable_if, doris::Status>::type std::__invoke_r(doris::pipeline::AggLocalState::init(doris::RuntimeState*, doris::pipeline::LocalStateInfo&)::$_2&, doris::RuntimeState*&&, doris::vectorized::Block*&&, bool*&&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9
apache#14 0x558c68b3e3e7 in std::_Function_handler::_M_invoke(std::_Any_data const&, doris::RuntimeState*&&, doris::vectorized::Block*&&, bool*&&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9
apache#15 0x558c68a8b819 in std::function::operator()(doris::RuntimeState*, doris::vectorized::Block*, bool*) const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9
apache#16 0x558c68a8b819 in doris::pipeline::AggSourceOperatorX::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/aggregation_source_operator.cpp:448:5
apache#17 0x558c68f652cb in doris::pipeline::PartitionedAggSourceOperatorX::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/partitioned_aggregation_source_operator.cpp:167:36
apache#18 0x558c676b1ddd in doris::pipeline::OperatorXBase::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/exec/operator.cpp:391:18
apache#19 0x558c69b0493d in doris::pipeline::PipelineTask::execute(bool*) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/pipeline_task.cpp:521:13
apache#20 0x558c69b52916 in doris::pipeline::TaskScheduler::_do_work(int) /home/zcp/repo_center/doris_master/doris/be/src/pipeline/task_scheduler.cpp:147:9
apache#21 0x558c54fa66f2 in doris::ThreadPool::dispatch_thread() /home/zcp/repo_center/doris_master/doris/be/src/util/threadpool.cpp:614:24
apache#22 0x558c54f83766 in std::function::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9
apache#23 0x558c54f83766 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_master/doris/be/src/util/thread.cpp:460:5
apache#24 0x558c507cbd26 in asan_thread_start(void*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x1f57fd26)
apache#25 0x7fb97f744ac2 in start_thread nptl/pthread_create.c:442:8
apache#26 0x7fb97f7d684f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/zcp/repo_center/doris_master/doris/be/src/util/counts.h:127:85
```
### What problem does this PR solve? introduced by apache#56405 2 changes here: 1. print local runtime filter mgr info inside each task 2. make a pair for tasks with its corresponding runtime state. ==198869==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7ba3927ac338 at pc 0x559d81e109b9 bp 0x7b7a5b8ad410 sp 0x7b7a5b8ad408 12:38:06 READ of size 8 at 0x7ba3927ac338 thread T1845 (EvHttpServer [w) 12:38:06 #0 0x559d81e109b8 in std::__uniq_ptr_impl<doris::RuntimeState, std::default_delete<doris::RuntimeState> >::_M_ptr() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/unique_ptr.h:193:51 12:38:06 #1 0x559d81e109b8 in std::unique_ptr<doris::RuntimeState, std::default_delete<doris::RuntimeState> >::get() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/unique_ptr.h:473:21 12:38:06 #2 0x559d81e109b8 in std::unique_ptr<doris::RuntimeState, std::default_delete<doris::RuntimeState> >::operator->() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/unique_ptr.h:466:9 12:38:06 #3 0x559d81e109b8 in doris::pipeline::PipelineFragmentContext::debug_string[abi:cxx11]() /root/doris/be/src/pipeline/pipeline_fragment_context.cpp:1941:28 12:38:06 apache#4 0x559d6d65aa2f in doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0::operator()(phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) const /root/doris/be/src/runtime/fragment_mgr.cpp:807:36 12:38:06 + echo 'cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/bin /home/work/pipline/backup_center/56368_04d746711764283b1cbafe40b1f849e10a5bcb6e_p0/fe/' 12:38:06 apache#5 0x559d6d65aa2f in doris::Status std::__invoke_impl<doris::Status, doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&>(std::__invoke_other, doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 12:38:06 + cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/conf /home/work/pipline/backup_center/56368_04d746711764283b1cbafe40b1f849e10a5bcb6e_p0/fe/ 12:38:06 apache#6 0x559d6d65aa2f in std::enable_if<is_invocable_r_v<doris::Status, doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&>, doris::Status>::type std::__invoke_r<doris::Status, doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&>(doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 12:38:06 apache#7 0x559d6d65aa2f in std::_Function_handler<doris::Status (phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&), doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long)::$_0>::_M_invoke(std::_Any_data const&, phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 12:38:06 apache#8 0x559d6d665d37 in std::function<doris::Status (phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&)>::operator()(phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&) const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 12:38:06 apache#9 0x559d6d665d37 in doris::ConcurrentContextMap<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, doris::pipeline::PipelineFragmentContext>::apply(std::function<doris::Status (phmap::flat_hash_map<std::pair<doris::TUniqueId, int>, std::shared_ptr<doris::pipeline::PipelineFragmentContext>, phmap::Hash<std::pair<doris::TUniqueId, int> >, phmap::EqualTo<std::pair<doris::TUniqueId, int> >, std::allocator<std::pair<std::pair<doris::TUniqueId, int> const, std::shared_ptr<doris::pipeline::PipelineFragmentContext> > > >&)>&&) /root/doris/be/src/runtime/fragment_mgr.h:94:31 12:38:06 apache#10 0x559d6d647f93 in doris::FragmentMgr::dump_pipeline_tasks[abi:cxx11](long) /root/doris/be/src/runtime/fragment_mgr.cpp:792:23 12:38:06 apache#11 0x559d7068942b in doris::LongPipelineTaskAction::handle(doris::HttpRequest*) /root/doris/be/src/http/action/pipeline_task_action.cpp:54:69 12:38:06 apache#12 0x559d82c776c6 (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x385416c6) 12:38:06 apache#13 0x559d82c5775f in bufferevent_run_readcb_ /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/bufferevent.c:251:3 12:38:06 apache#14 0x559d82c79802 in bufferevent_trigger_nolock_ /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/bufferevent-internal.h:411:3 12:38:06 apache#15 0x559d82c79802 in bufferevent_readcb /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/bufferevent_sock.c:214:2 12:38:06 apache#16 0x559d82c60724 in event_persist_closure /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/event.c:1623:9 12:38:06 apache#17 0x559d82c60724 in event_process_active_single_queue /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/event.c:1682:4 12:38:06 apache#18 0x559d82c60d76 in event_process_active /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/event.c:1783:9 12:38:06 + echo 'cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/log /home/work/pipline/backup_center/56368_04d746711764283b1cbafe40b1f849e10a5bcb6e_p0/fe/' 12:38:06 apache#19 0x559d82c6355f in event_base_loop.constprop.0 /home/runner/work/doris-thirdparty/doris-thirdparty/thirdparty/src/libevent-release-2.1.12-stable/event.c:2006:12 12:38:06 apache#20 0x559d7072b08c in doris::EvHttpServer::start()::$_0::operator()() const /root/doris/be/src/http/ev_http_server.cpp:139:13 12:38:06 + cp -r /mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0//Cluster0/fe/log /home/work/pipline/backup_center/56368_04d746711764283b1cbafe40b1f849e10a5bcb6e_p0/fe/ 12:38:06 apache#21 0x559d7072b08c in void std::__invoke_impl<void, doris::EvHttpServer::start()::$_0&>(std::__invoke_other, doris::EvHttpServer::start()::$_0&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 12:38:06 apache#22 0x559d7072b08c in std::enable_if<is_invocable_r_v<void, doris::EvHttpServer::start()::$_0&>, void>::type std::__invoke_r<void, doris::EvHttpServer::start()::$_0&>(doris::EvHttpServer::start()::$_0&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113:2 12:38:06 apache#23 0x559d7072b08c in std::_Function_handler<void (), doris::EvHttpServer::start()::$_0>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 12:38:06 apache#24 0x559d6e121772 in doris::ThreadPool::dispatch_thread() /root/doris/be/src/util/threadpool.cpp:614:24 12:38:06 apache#25 0x559d6e0fe7e6 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 12:38:06 apache#26 0x559d6e0fe7e6 in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:460:5 12:38:06 apache#27 0x559d6984ed26 in asan_thread_start(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P0/Cluster0/be/lib/doris_be+0x1f118d26) 12:38:06 apache#28 0x7f83909ee608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8 12:38:06 apache#29 0x7f8390901132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95 ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…sMgr::report_runtime_query_statistics (apache#56744) try to avoid asan check fail: ```cpp OpenJDK 64-Bit Server VM warning: Option CriticalJNINatives was deprecated in version 16.0 and will likely be removed in a future release. Listening for transport dt_socket at address: 8871 WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance. start BE in local mode thrift error, reason=write() send(): Broken pipe================================================================= AddressSanitizer: CHECK failed: asan_thread.cpp:369 "((ptr[0] == kCurrentStackFrameMagic)) != (0)" (0x0, 0x0) (tid=13525) #0 0x557a503468e1 in __asan::CheckUnwind() (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2a08e1) #1 0x557a50361182 in __sanitizer::CheckFailed(char const*, int, char const*, unsigned long long, unsigned long long) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2bb182) #2 0x557a5034a97b in __asan::AsanThread::GetStackFrameAccessByAddr(unsigned long, __asan::AsanThread::StackFrameAccess*) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2a497b) #3 0x557a502a4a30 in __asan::AddressDescription::AddressDescription(unsigned long, unsigned long, bool) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f1fea30) apache#4 0x557a502a67a3 in __asan::ErrorGeneric::ErrorGeneric(unsigned int, unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f2007a3) apache#5 0x557a50343878 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f29d878) apache#6 0x557a50344f75 in __asan_report_load8 (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f29ef75) apache#7 0x557a5039791e in std::__cxx11::basic_string, std::allocator>::_M_create(unsigned long&, unsigned long) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.tcc:148:11 apache#8 0x557a503984db in void std::__cxx11::basic_string, std::allocator>::_M_construct(char const*, unsigned long) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.tcc:291:12 apache#9 0x557a503984db in std::__cxx11::basic_string, std::allocator>::basic_string(std::__cxx11::basic_string, std::allocator> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.h:617:2 apache#10 0x557a55b4d77d in std::pair, std::allocator> const, doris::TQueryStatistics>::pair(std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_pair.h:312:17 apache#11 0x557a55b4d77d in _ZSt12construct_atISt4pairIKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEN5doris16TQueryStatisticsEEJRKSA_EQaant20is_unbounded_array_vIT_ErqXgsnwcvPvLi0E_SD_pispclsr3stdE7declvalIT0_EEEEEPSD_SG_DpOSF_ /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_construct.h:110:22 apache#12 0x557a55b4d77d in void std::allocator_traits, std::allocator> const, doris::TQueryStatistics>>>>::construct, std::allocator> const, doris::TQueryStatistics>, std::pair, std::allocator> const, doris::TQueryStatistics> const&>(std::allocator, std::allocator> const, doris::TQueryStatistics>>>&, std::pair, std::allocator> const, doris::TQueryStatistics>*, std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/alloc_traits.h:676:4 apache#13 0x557a55b4d77d in void std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_construct_node, std::allocator> const, doris::TQueryStatistics> const&>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1237:8 apache#14 0x557a55b4cfe3 in std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node::operator(), std::allocator> const, doris::TQueryStatistics> const&>(std::pair, std::allocator> const, doris::TQueryStatistics> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h apache#15 0x557a55b4cbe6 in std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_clone_node, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1287:8 apache#16 0x557a55b4c6f6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2546:20 apache#17 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3 apache#18 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3 apache#19 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3 apache#20 0x557a55b4c7b6 in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree_node, std::allocator> const, doris::TQueryStatistics>>*, std::_Rb_tree_node_base*, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2554:3 apache#21 0x557a55b4c27a in std::_Rb_tree_node_base* std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_M_copy, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node>(std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&, std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::_Reuse_or_alloc_node&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:1523:6 apache#22 0x557a55740d4f in std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::operator=(std::_Rb_tree, std::allocator>, std::pair, std::allocator> const, doris::TQueryStatistics>, std::_Select1st, std::allocator> const, doris::TQueryStatistics>>, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_tree.h:2453:18 apache#23 0x557a55740d4f in std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>>::operator=(std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/stl_map.h:354:37 apache#24 0x557a55740d4f in doris::TReportWorkloadRuntimeStatusParams::__set_query_statistics_map(std::map, std::allocator>, doris::TQueryStatistics, std::less, std::allocator>>, std::allocator, std::allocator> const, doris::TQueryStatistics>>> const&) /home/zcp/repo_center/doris_branch-4.0/doris/gensrc/build/gen_cpp/FrontendService_types.cpp:4391:30 apache#25 0x557a5447e13a in doris::RuntimeQueryStatisticsMgr::report_runtime_query_statistics() /home/zcp/repo_center/doris_branch-4.0/doris/be/src/runtime/runtime_query_statistics_mgr.cpp:403:31 apache#26 0x557a504f1192 in doris::Daemon::report_runtime_query_statistics_thread() /home/zcp/repo_center/doris_branch-4.0/doris/be/src/common/daemon.cpp:457:65 apache#27 0x557a54bf2eb6 in std::function::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 apache#28 0x557a54bf2eb6 in doris::Thread::supervise_thread(void*) /home/zcp/repo_center/doris_branch-4.0/doris/be/src/util/thread.cpp:460:5 apache#29 0x557a50336d26 in asan_thread_start(void*) (/mnt/hdd01/ci/doris-deploy-branch-4.0-local/be/lib/doris_be+0x1f290d26) apache#30 0x7f12a399eac2 in start_thread nptl/pthread_create.c:442:8 apache#31 0x7f12a3a3084f in __closefrom_fallback misc/../sysdeps/unix/sysv/linux/closefrom_fallback.c:45:19 172.20.59.34 last coredump sql: last SQL query not found ```
…e#56935) ``` (gdb) bt #0 doris::cloud::FdbTxnKv::get_partition_boundaries (this=0x7f11ac6e3950, boundaries=0x7f1130017f20) at /home/zcp/repo_center/doris_branch-4.0/doris/cloud/src/meta-store/txn_kv.cpp:212 #1 0x000055e494de9276 in doris::cloud::export_fdb_kv_ranges_details (kv=0x7f11ac6e3950) at /home/zcp/repo_center/doris_branch-4.0/doris/cloud/src/common/metric.cpp:358 #2 doris::cloud::FdbMetricExporter::export_fdb_metrics (txn_kv=0x7f11ac6e3950) at /home/zcp/repo_center/doris_branch-4.0/doris/cloud/src/common/metric.cpp:393 #3 0x000055e494dec77d in doris::cloud::FdbMetricExporter::start()::$_0::operator()() const (this=<optimized out>) at /home/zcp/repo_center/doris_branch-4.0/doris/cloud/src/common/metric.cpp:415 apache#4 std::__invoke_impl<void, doris::cloud::FdbMetricExporter::start()::$_0>(std::__invoke_other, doris::cloud::FdbMetricExporter::start()::$_0&&) (__f=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#5 std::__invoke<doris::cloud::FdbMetricExporter::start()::$_0>(doris::cloud::FdbMetricExporter::start()::$_0&&) (__fn=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:98 apache#6 std::thread::_Invoker<std::tuple<doris::cloud::FdbMetricExporter::start()::$_0> >::_M_invoke<0ul>(std::_Index_tuple<0ul>) (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:303 apache#7 std::thread::_Invoker<std::tuple<doris::cloud::FdbMetricExporter::start()::$_0> >::operator()() (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:310 apache#8 std::thread::_State_impl<std::thread::_Invoker<std::tuple<doris::cloud::FdbMetricExporter::start()::$_0> > >::_M_run() (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_thread.h:255 apache#9 0x000055e495e547b0 in execute_native_thread_routine () apache#10 0x00007f11b1dfeac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#11 0x00007f11b1e90850 in __closefrom_fallback (from=-1266337808, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#12 0x0000000000000000 in ?? () ```
…56755) ### What problem does this PR solve? ```text #0 __GI___pthread_sigmask (how=2, newmask=<optimized out>, oldmask=0x0) at ./nptl/pthread_sigmask.c:43 #1 0x00007f91fd6c471e in PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #2 0x00007f91fd6c5206 in JVM_handle_linux_signal () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #3 <signal handler called> apache#4 __gnu_cxx::__exchange_and_add (__mem=0xe, __val=-1) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/ext/atomicity.h:68 apache#5 __gnu_cxx::__exchange_and_add_dispatch (__mem=0xe, __val=-1) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/ext/atomicity.h:103 apache#6 std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_weak_release (this=0x2) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:211 apache#7 std::__weak_count<(__gnu_cxx::_Lock_policy)2>::~__weak_count (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1168 apache#8 std::__weak_ptr<doris::QueryContext, (__gnu_cxx::_Lock_policy)2>::~__weak_ptr (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:2003 apache#9 doris::QueryTaskController::revoke_memory()::$_1::~$_1() (this=<optimized out>) at /root/doris/be/src/runtime/workload_management/query_task_controller.cpp:168 apache#10 std::_Function_base::_Base_manager<doris::QueryTaskController::revoke_memory()::$_1>::_M_destroy(std::_Any_data&, std::integral_constant<bool, false>) (__victim=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:177 apache#11 std::_Function_base::_Base_manager<doris::QueryTaskController::revoke_memory()::$_1>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (__dest=..., __op=std::__destroy_functor, __source=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:205 apache#12 std::_Function_handler<void (doris::pipeline::SpillContext*), doris::QueryTaskController::revoke_memory()::$_1>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) (__dest=..., __source=..., __op=std::__destroy_functor) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:284 apache#13 0x000055f37fba60c5 in std::_Function_base::~_Function_base (this=0x7f8dbb0c9fe0) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:246 apache#14 doris::pipeline::SpillContext::~SpillContext (this=0x7f91536f0150) at /root/doris/be/src/pipeline/exec/spill_utils.h:57 apache#15 0x000055f384259e4b in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x7f91536f0140) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:345 apache#16 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::operator= (this=0x7f8d61302518, __r=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1088 apache#17 std::__shared_ptr<doris::pipeline::SpillContext, (__gnu_cxx::_Lock_policy)2>::operator= (this=0x7f8d61302510) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1530 apache#18 std::shared_ptr<doris::pipeline::SpillContext>::operator= (this=0x7f8d61302510) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr.h:413 apache#19 doris::pipeline::PipelineTask::revoke_memory (this=0x7f8d61302490, spill_context=...) at /root/doris/be/src/pipeline/pipeline_task.cpp:812 apache#20 0x000055f37fba3c2d in doris::QueryTaskController::revoke_memory (this=<optimized out>) at /root/doris/be/src/runtime/workload_management/query_task_controller.cpp:185 apache#21 0x000055f37fb99d08 in doris::WorkloadGroupMgr::handle_single_query_ (this=<optimized out>, requestor=..., size_to_reserve=size_to_reserve@entry=1024000, time_in_queue=time_in_queue@entry=27, paused_reason=...) at /root/doris/be/src/runtime/workload_group/workload_group_manager.cpp:820 apache#22 0x000055f37fb981d0 in doris::WorkloadGroupMgr::handle_paused_queries (this=0x7f9141d8a800) at /root/doris/be/src/runtime/workload_group/workload_group_manager.cpp:381 apache#23 0x000055f37eb97297 in doris::Daemon::memory_maintenance_thread (this=0x7ffe63cad730) at /root/doris/be/src/common/daemon.cpp:354 apache#24 0x000055f37fd812fc in std::function<void ()>::operator()() const (this=0x7f8dbb0c9fe0) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#25 doris::Thread::supervise_thread (arg=0x7f914ba25e10) at /root/doris/be/src/util/thread.cpp:460 apache#26 0x00007f91fc75fac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#27 0x00007f91fc7f1850 in __closefrom_fallback (from=1674236160, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#28 0x0000000000000000 in ?? () ``` Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…icate (apache#57100) ### What problem does this PR solve? ```text #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=140649357411904) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=140649357411904) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=140649357411904, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x00007feca2a42476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 apache#4 0x00007feca2a287f3 in __GI_abort () at ./stdlib/abort.c:79 apache#5 0x000055cd63341b52 in __gnu_cxx::__verbose_terminate_handler() () apache#6 0x000055cd6333fe26 in __cxxabiv1::__terminate(void (*)()) () apache#7 0x000055cd6333fe79 in std::terminate() () apache#8 0x000055cd6333ffc3 in __cxa_throw () apache#9 0x000055cd6339556a in std::__throw_out_of_range(char const*) () apache#10 0x000055cd5a9a7a68 in __gnu_cxx::__stoa<double, double, char> (__str=0x7fe842726fc0 "9.", '9' <repeats 15 times>, "1e-309", __idx=0x0, __convf=<optimized out>, __name=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/ext/string_conversions.h:90 apache#11 std::__cxx11::stod (__str=..., __idx=0x0) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/basic_string.h:4490 apache#12 doris::IntegerPredicateCreator<(doris::PrimitiveType)9, (doris::PredicateType)1, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::convert (condition=...) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:68 apache#13 doris::IntegerPredicateCreator<(doris::PrimitiveType)9, (doris::PredicateType)1, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::create (this=<optimized out>, column=..., index=1, conditions=..., opposite=false, arena=...) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:59 apache#14 0x000055cd5a3dd63b in doris::create_predicate<(doris::PredicateType)1, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > (column=..., index=1, conditions=..., opposite=false, arena=...) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:270 apache#15 doris::create_comparison_predicate<(doris::PredicateType)1> (column=..., index=1, condition=..., opposite=false, arena=...) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:278 apache#16 0x000055cd5a3dd435 in doris::parse_to_predicate (column=..., index=1, condition=..., arena=..., opposite=false) at /root/doris_branch-4.0/doris/be/src/olap/predicate_creator.h:326 apache#17 0x000055cd5afc2759 in doris::TabletReader::_init_conditions_param (this=this@entry=0x7fe842e1a800, read_params=...) at /root/doris_branch-4.0/doris/be/src/olap/tablet_reader.cpp:549 apache#18 0x000055cd5afc1022 in doris::TabletReader::_init_params (this=this@entry=0x7fe842e1a800, read_params=...) at /root/doris_branch-4.0/doris/be/src/olap/tablet_reader.cpp:304 apache#19 0x000055cd5afbfe00 in doris::TabletReader::init (this=0x7fe842e1a800, read_params=...) at /root/doris_branch-4.0/doris/be/src/olap/tablet_reader.cpp:127 apache#20 0x000055cd5f019d0f in doris::vectorized::BlockReader::init (this=0x4c1b, read_params=...) at /root/doris_branch-4.0/doris/be/src/vec/olap/block_reader.cpp:206 apache#21 0x000055cd5f79a3f4 in doris::vectorized::OlapScanner::open (this=0x7febe2c16010, state=<optimized out>) at /root/doris_branch-4.0/doris/be/src/vec/exec/scan/olap_scanner.cpp:278 apache#22 0x000055cd5e05303a in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=..., scan_task=...) at /root/doris_branch-4.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:182 apache#23 0x000055cd5e0549e5 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const (this=<optimized out>) at /root/doris_branch-4.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:96 apache#24 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}::operator()() const (this=0x7febe321dac0) at /root/doris_branch-4.0/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:95 apache#25 std::__invoke_impl<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&) (__f=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#26 std::__invoke_r<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&) (__fn=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116 apache#27 std::_Function_handler<bool (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) ( __functor=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292 apache#28 0x000055cd5b4ab5d5 in doris::ThreadPool::dispatch_thread (this=0x7fec35e01800) at /root/doris_branch-4.0/doris/be/src/util/threadpool.cpp:614 apache#29 0x000055cd5b4a06fc in std::function<void ()>::operator()() const (this=0x4a17) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#30 doris::Thread::supervise_thread (arg=0x7fec35872110) at /root/doris_branch-4.0/doris/be/src/util/thread.cpp:460 apache#31 0x00007feca2a94ac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#32 0x00007feca2b26850 in __closefrom_fallback (from=-823527648, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#33 0x0000000000000000 in ?? () ``` Issue Number: close #xxx Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
### What problem does this PR solve? ```text #0 __pthread_kill_implementation (no_tid=0, signo=6, threadid=136182904088128) at ./nptl/pthread_kill.c:44 #1 __pthread_kill_internal (signo=6, threadid=136182904088128) at ./nptl/pthread_kill.c:78 #2 __GI___pthread_kill (threadid=136182904088128, signo=signo@entry=6) at ./nptl/pthread_kill.c:89 #3 0x00007fdee8ffb476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26 apache#4 0x00007fdee8fe17f3 in __GI_abort () at ./stdlib/abort.c:79 apache#5 0x000055cc31fb9bcd in ?? () apache#6 0x000055cc31fadeac in google::LogMessage::SendToLog() () apache#7 0x000055cc31fae538 in google::LogMessage::Flush() () apache#8 0x000055cc31fb2116 in google::LogMessageFatal::~LogMessageFatal() () apache#9 0x000055cc1977f798 in doris::vectorized::PODArray<unsigned int, 4096ul, doris::Allocator<false, false, false, doris::DefaultMemoryAllocator, false>, 16ul, 15ul>::operator[] (this=<optimized out>, n=2) at /root/doris/be/src/vec/common/pod_array.h:466 apache#10 0x000055cc19777580 in doris::vectorized::ColumnStr<unsigned int>::size_at (this=0x7c5eea94dda0, i=2) at /root/doris/be/src/vec/columns/column_string.h:96 apache#11 doris::vectorized::ColumnStr<unsigned int>::compare_at (this=0x7c5eea94dda0, n=2, m=1, rhs_=...) at /root/doris/be/src/vec/columns/column_string.h:482 apache#12 0x000055cc2f2f1138 in doris::vectorized::VCollectIterator::_topn_next (this=<optimized out>, block=<optimized out>) at /root/doris/be/src/vec/olap/vcollect_iterator.cpp:343 apache#13 0x000055cc2f2efa08 in doris::vectorized::VCollectIterator::next (this=0x898f, block=0x6) at /root/doris/be/src/vec/olap/vcollect_iterator.cpp:247 apache#14 0x000055cc2f28e3ea in doris::vectorized::BlockReader::_direct_next_block (this=0x7dbeed736880, block=<optimized out>, eof=<optimized out>) at /root/doris/be/src/vec/olap/block_reader.cpp:267 apache#15 0x000055cc2f28778b in doris::vectorized::BlockReader::next_block_with_aggregation (this=0x7dbeed736880, block=<optimized out>, eof=<optimized out>) at /root/doris/be/src/vec/olap/block_reader.cpp:66 apache#16 0x000055cc2b33300e in doris::vectorized::OlapScanner::_get_block_impl (this=0x7daeee690690, state=<optimized out>, block=0x7daeee6907b8, eof=<optimized out>) at /root/doris/be/src/vec/exec/scan/olap_scanner.cpp:584 apache#17 0x000055cc2b2c990d in doris::vectorized::Scanner::get_block (this=<optimized out>, state=<optimized out>, block=0x7daeee6907b8, eof=<optimized out>) at /root/doris/be/src/vec/exec/scan/scanner.cpp:116 apache#18 0x000055cc2b2c8fa0 in doris::vectorized::Scanner::get_block_after_projects (this=0x7daeee690690, state=0x7dcee9e6ec80, block=0x7caee9b74fe0, eos=0x7bdb809a11f0) at /root/doris/be/src/vec/exec/scan/scanner.cpp:82 apache#19 0x000055cc2b2e51b4 in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=..., scan_task=...) at /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:182 apache#20 0x000055cc2b2ed92c in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const (this=<optimized out>) at /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:96 apache#21 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}::operator()() const (this=0x7c0eeec42540) at /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:95 apache#22 std::__invoke_impl<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&) (__f=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#23 std::__invoke_r<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}&) (__fn=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116 apache#24 std::_Function_handler<bool (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) ( __functor=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292 apache#25 0x000055cc1dd09d33 in doris::ThreadPool::dispatch_thread (this=0x7d3ee8680f80) at /root/doris/be/src/util/threadpool.cpp:614 apache#26 0x000055cc1dce6da7 in std::function<void ()>::operator()() const (this=<optimized out>) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#27 doris::Thread::supervise_thread (arg=<optimized out>) at /root/doris/be/src/util/thread.cpp:460 apache#28 0x000055cc19364d27 in asan_thread_start(void*) () apache#29 0x00007fdee904dac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#30 0x00007fdee90df850 in __closefrom_fallback (from=877593024, dirfd_fallback=<optimized out>) at ../sysdeps/unix/sysv/linux/closefrom_fallback.c:45 apache#31 0x0000000000000000 in ?? () ``` Related PR: #xxx Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…ntains window function (apache#55066) ### What problem does this PR solve? Support window function rewrite when materialized view contains window function Such as mv def is as following: CREATE MATERIALIZED VIEW mv1 BUILD IMMEDIATE REFRESH COMPLETE ON MANUAL DISTRIBUTED BY RANDOM BUCKETS 2 PROPERTIES ('replication_num' = '1') AS select * from ( select o_orderkey, FIRST_VALUE(o_custkey) OVER ( PARTITION BY o_orderdate ORDER BY o_totalprice NULLS LAST RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ) AS first_value, RANK() OVER ( PARTITION BY o_orderdate, o_orderstatus ORDER BY o_totalprice NULLS LAST RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ) AS rank_value, LAG(l_extendedprice, 1, 0) over (partition by o_orderdate, l_shipdate order by l_quantity) AS lag_value from lineitem2 left join orders2 on l_orderkey = o_orderkey and l_shipdate = o_orderdate ) t where o_orderkey > 1; if query as following, this can use mv to represent query select * from ( select o_orderkey, FIRST_VALUE(o_custkey) OVER ( PARTITION BY o_orderdate ORDER BY o_totalprice NULLS LAST RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ) AS first_value, RANK() OVER ( PARTITION BY o_orderdate, o_orderstatus ORDER BY o_totalprice NULLS LAST RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW ) AS rank_value, LAG(l_extendedprice, 1, 0) over (partition by o_orderdate, l_shipdate order by l_quantity) AS lag_value from lineitem2 left join orders2 on l_orderkey = o_orderkey and l_shipdate = o_orderdate ) t where o_orderkey > 2; explain result is as follwing: +----------------------------------------------------------------------------------------+ | Explain String(Nereids Planner) | +----------------------------------------------------------------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | o_orderkey[apache#4] | | first_value[apache#5] | | rank_value[apache#6] | | lag_value[apache#7] | | PARTITION: RANDOM | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCAL | | | | 0:VOlapScanNode(445) | | TABLE: regression_test_nereids_rules_p0_mv_window.mv1(mv1), PREAGGREGATION: ON | | PREDICATES: (o_orderkey[#0] > 2) | | partitions=1/1 (mv1) | | tablets=2/2, tabletList=1755678381149,1755678381151 | | cardinality=1, avgRowSize=0.0, numNodes=1 | | pushAggOp=NONE | | final projections: o_orderkey[#0], first_value[#1], rank_value[#2], lag_value[#3] | | final project output tuple id: 1 | | | | | | ========== MATERIALIZATIONS ========== | | | | MaterializedView | | MaterializedViewRewriteSuccessAndChose: | | internal.regression_test_nereids_rules_p0_mv_window.mv1 chose | | | | MaterializedViewRewriteSuccessButNotChose: | | | | MaterializedViewRewriteFail: | | | | | | ========== STATISTICS ========== | | planed with unknown column statistics | +----------------------------------------------------------------------------------------+
…ich belongs to an agg materialized view (apache#58038) ### What problem does this PR solve? Issue Number: close apache#58037 Problem Summary: ``` #0 0x00007f9aca4a3f8c in __pthread_kill_implementation () from /lib64/libc.so.6 #1 0x00007f9aca454a26 in raise () from /lib64/libc.so.6 #2 0x00007f9aca43d87c in abort () from /lib64/libc.so.6 #3 0x0000561dc3d1ea1d in ?? () apache#4 0x0000561dc3d1105a in google::LogMessage::Fail() () apache#5 0x0000561dc3d14146 in google::LogMessage::SendToLog() () apache#6 0x0000561dc3d10b90 in google::LogMessage::Flush() () apache#7 0x0000561dc3d14989 in google::LogMessageFatal::~LogMessageFatal() () apache#8 0x0000561db854c996 in assert_cast<doris::vectorized::ColumnStr<unsigned int> const&, (TypeCheckOnRelease)1, doris::vectorized::IColumn const&>(doris::vectorized::IColumn const&)::{lambda(auto:1&&)#1}::operator()<doris::vectorized::IColumn const&>(doris::vectorized::IColumn const&) const (this=this@entry=0x7f9658ccc1f8, from=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/common/assert_cast.h:58 apache#9 0x0000561db854c7d7 in assert_cast<doris::vectorized::ColumnStr<unsigned int> const&, (TypeCheckOnRelease)1, doris::vectorized::IColumn const&> (from=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/common/assert_cast.h:73 apache#10 0x0000561db854bb0b in doris::vectorized::ColumnStr<unsigned int>::compare_at (this=0x7f957a14e2c0, n=1159288, m=6, rhs_=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/columns/column_string.h:526 apache#11 0x0000561dbe108c6b in doris::vectorized::GenericComparisonImpl<doris::vectorized::EqualsOp<int, int> >::vector_constant (a=..., b=..., c=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/functions_comparison.h:112 apache#12 doris::vectorized::FunctionComparison<doris::vectorized::EqualsOp, doris::vectorized::NameEquals>::execute_generic_identical_types ( this=<optimized out>, block=..., result=result@entry=10, c0=0x7f957a14e2c0, c1=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/functions_comparison.h:506 apache#13 0x0000561dbdf9e97e in doris::vectorized::FunctionComparison<doris::vectorized::EqualsOp, doris::vectorized::NameEquals>::execute_generic ( this=0x7f96d6fb1b90, block=..., result=10, c0=..., c1=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/functions_comparison.h:517 apache#14 doris::vectorized::FunctionComparison<doris::vectorized::EqualsOp, doris::vectorized::NameEquals>::execute_impl (this=0x7f96d6fb1b90, context=<optimized out>, block=..., arguments=..., result=10, input_rows_count=104) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/functions_comparison.h:707 apache#15 0x0000561dbdcf1b8f in doris::vectorized::DefaultExecutable::execute_impl (this=<optimized out>, context=0x6, block=..., arguments=..., result=1, input_rows_count=104) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/function.h:472 apache#16 0x0000561dbeea76ae in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal (this=this@entry=0x7f99f62a65d0, context=context@entry=0x7f99f6442b00, block=..., args=..., result=result@entry=10, input_rows_count=104, dry_run=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/function.cpp:121 apache#17 0x0000561dbeea4ce8 in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns (this=0x7f99f62a65d0, context=0x7f99f6442b00, block=..., args=..., result=10, input_rows_count=104, dry_run=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/function.cpp:246 apache#18 doris::vectorized::PreparedFunctionImpl::default_implementation_for_nulls (this=this@entry=0x7f99f62a65d0, context=context@entry=0x7f99f6442b00, block=..., args=..., result=result@entry=10, input_rows_count=104, dry_run=<optimized out>, executed=0x7f9658ccc666) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/function.cpp:218 apache#19 0x0000561dbeea4e9c in doris::vectorized::PreparedFunctionImpl::_execute_skipped_constant_deal (this=0x7f99f62a65d0, context=0x7f99f6442b00, block=..., args=..., result=10, input_rows_count=<optimized out>, dry_run=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/function.cpp:112 apache#20 doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns (this=0x7f99f62a65d0, context=0x7f99f6442b00, block=..., args=..., result=10, input_rows_count=104, dry_run=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/function.cpp:246 apache#21 0x0000561dbeea4f66 in doris::vectorized::PreparedFunctionImpl::execute (this=0x11b078, context=0x6, block=..., args=..., result=1, input_rows_count=104, dry_run=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/function.cpp:252 apache#22 0x0000561dbdcf1500 in doris::vectorized::IFunctionBase::execute (this=<optimized out>, context=0x7f99f6442b00, block=..., arguments=..., result=10, input_rows_count=104, dry_run=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/functions/function.h:195 --Type <RET> for more, q to quit, c to continue without paging--c apache#23 0x0000561dbdceccad in doris::vectorized::VectorizedFnCall::_do_execute (this=0x7f96f0fec510, context=0x7f957f09cdf0, block=0x7f957b02a3b0, result_column_id=0x7f9658ccca14, args=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exprs/vectorized_fn_call.cpp:197 apache#24 0x0000561dbdced2c6 in doris::vectorized::VectorizedFnCall::execute (this=0x11b078, context=0x6, block=0x7f9aca4a3f8c <__pthread_kill_implementation+268>, result_column_id=0x7f9658ccbe10) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exprs/vectorized_fn_call.cpp:212 apache#25 0x0000561dbdd1e51b in doris::vectorized::VExprContext::execute (this=0x7f957f09cdf0, block=0x7f9aca4a3f8c <__pthread_kill_implementation+268>, block@entry=0x7f957b02a3b0, result_column_id=result_column_id@entry=0x7f9658ccca14) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exprs/vexpr_context.cpp:55 apache#26 0x0000561dbdd1fcb5 in doris::vectorized::VExprContext::execute_conjuncts (ctxs=..., filters=filters@entry=0x0, accept_null=false, block=block@entry=0x7f957b02a3b0, result_filter=result_filter@entry=0x7f9658ccccc0, can_filter_all=0x7f9658cccbc7) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exprs/vexpr_context.cpp:174 apache#27 0x0000561dbdd2131f in doris::vectorized::VExprContext::execute_conjuncts_and_filter_block (ctxs=..., block=0x7f957b02a3b0, columns_to_filter=..., column_to_keep=6, filter=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exprs/vexpr_context.cpp:354 apache#28 0x0000561db8f49450 in doris::segment_v2::SegmentIterator::_execute_common_expr (this=this@entry=0x7f954e20a000, sel_rowid_idx=0x7f955c0d2000, selected_size=@0x7f9658ccce6e: 104, block=block@entry=0x7f957b02a3b0) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2338 apache#29 0x0000561db8f482f8 in doris::segment_v2::SegmentIterator::_next_batch_internal (this=0x7f954e20a000, block=0x7f957b02a3b0) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:2230 apache#30 0x0000561db8f45212 in doris::segment_v2::SegmentIterator::next_batch(doris::vectorized::Block*)::$_0::operator()() const ( this=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1953 apache#31 doris::segment_v2::SegmentIterator::next_batch (this=0x7f954e20a000, block=0x6) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/olap/rowset/segment_v2/segment_iterator.cpp:1952 apache#32 0x0000561db8ee49bc in doris::segment_v2::LazyInitSegmentIterator::next_batch (this=0x7f953bc71f80, block=0x7f957b02a3b0) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/olap/rowset/segment_v2/lazy_init_segment_iterator.h:44 apache#33 0x0000561db8dab844 in doris::BetaRowsetReader::next_block (this=0x7f9a4e215800, block=0x7f957b02a3b0) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/olap/rowset/beta_rowset_reader.cpp:377 apache#34 0x0000561dc2c9413d in doris::vectorized::VCollectIterator::Level0Iterator::_refresh (this=0x7f953ba137a0) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/olap/vcollect_iterator.h:256 apache#35 doris::vectorized::VCollectIterator::Level0Iterator::refresh_current_row (this=this@entry=0x7f953ba137a0) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/olap/vcollect_iterator.cpp:509 apache#36 0x0000561dc2c93bf4 in doris::vectorized::VCollectIterator::Level0Iterator::init (this=0x7f953ba137a0, get_data_by_ref=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/olap/vcollect_iterator.cpp:461 apache#37 0x0000561dc2c91002 in doris::vectorized::VCollectIterator::build_heap (this=0x7f957a52bb30, rs_readers=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/olap/vcollect_iterator.cpp:125 apache#38 0x0000561dc2c7e1f2 in doris::vectorized::BlockReader::_init_collect_iter (this=this@entry=0x7f957a52b400, read_params=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/olap/block_reader.cpp:153 apache#39 0x0000561dc2c7f191 in doris::vectorized::BlockReader::init (this=<optimized out>, read_params=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/olap/block_reader.cpp:226 apache#40 0x0000561dc3937869 in doris::vectorized::NewOlapScanner::open (this=0x7f9a56270210, state=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exec/scan/new_olap_scanner.cpp:252 apache#41 0x0000561dbdcc5413 in doris::vectorized::ScannerScheduler::_scanner_scan (ctx=..., scan_task=...) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:221 apache#42 0x0000561dbdcc62bd in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const::{lambda()#1}::operator()() const (this=<optimized out>) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:154 apache#43 doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}::operator()() const (this=0x7f954e25d3c0) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:153 apache#44 std::__invoke_impl<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__f=...) at /data/home/lambxu/installs/ldb_toolchain_bak/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:61 apache#45 std::__invoke_r<void, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}&) (__fn=...) at /data/home/lambxu/installs/ldb_toolchain_bak/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/invoke.h:111 apache#46 std::_Function_handler<void (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_1::operator()() const::{lambda()#1}>::_M_invoke(std::_Any_data const&) (__functor=...) at /data/home/lambxu/installs/ldb_toolchain_bak/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:291 apache#47 0x0000561db962137a in doris::ThreadPool::dispatch_thread (this=0x7f9a50f9d380) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/util/threadpool.cpp:602 apache#48 0x0000561db96159a1 in std::function<void ()>::operator()() const (this=0x11a791) at /data/home/lambxu/installs/ldb_toolchain_bak/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/std_function.h:560 apache#49 doris::Thread::supervise_thread (arg=0x7f969f569ce0) at /data/home/lambxu/work/git/doris-3.1/doris/be/src/util/thread.cpp:498 apache#50 0x00007f9aca4a2215 in start_thread () from /lib64/libc.so.6 apache#51 0x00007f9aca524bdc in clone3 () from /lib64/libc.so.6 ``` Assume that 0,1,2,3,4, is key columns of an AGG mv, because the PreAgg is OFF at scan node, the block will contain all key columns to merge data in storage layer. if we select 0,1 column, with 3,4 column in where clause, then the slot ids should be 0,1,3,4, and column ids in conjuncts is the index of slot ids.(which is 2 and 3) But the plan use the key type of base table which is DUP key, treating the AGG mv as a DUP mv, so these conjuncts are pushed down to the scan node which belongs to an AGG mv, these conjuncts will pick the wrong column 2 and 3 (which shoud be 4 and 5) in block to exucute. So we should use the key type of mv but not the key type of base table. ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [x] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [x] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [x] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…57888) We previously had a crash. The cause is that we should not access the request after calling add_block(...) because add_block may enqueue a closure that runs on another thread and frees the request ``` ==730145==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7be1efd803a0 at pc 0x556b38d9d625 bp 0x7b16bf0193f0 sp 0x7b16bf0193e8 READ of size 4 at 0x7be1efd803a0 thread T1559 #0 0x556b38d9d624 in google::protobuf::internal::RepeatedPtrFieldBase::size() const /home/zcp/repo_center/doris_master/doris/thirdparty/installed/include/google/protobuf/repeated_ptr_field.h:185:29 #1 0x556b408ab062 in google::protobuf::RepeatedPtrField<doris::PBlock>::size() const /home/zcp/repo_center/doris_master/doris/thirdparty/installed/include/google/protobuf/repeated_ptr_field.h:1248:32 #2 0x556b408aaff4 in doris::PTransmitDataParams::_internal_blocks_size() const /home/zcp/repo_center/doris_master/doris/be/../gensrc/build/gen_cpp/internal_service.pb.h:32149:25 #3 0x556b4089731c in doris::PTransmitDataParams::blocks_size() const /home/zcp/repo_center/doris_master/doris/be/../gensrc/build/gen_cpp/internal_service.pb.h:32152:10 apache#4 0x556b60a83c17 in doris::vectorized::VDataStreamMgr::transmit_block(doris::PTransmitDataParams const*, google::protobuf::Closure**, long) /home/zcp/repo_center/doris_master/doris/be/src/vec/runtime/vdata_stream_mgr.cpp:150:38 apache#5 0x556b407f7408 in doris::PInternalService::_transmit_block(google::protobuf::RpcController*, doris::PTransmitDataParams const*, doris::PTransmitDataResult*, google::protobuf::Closure*, doris::Status const&, long) /home/zcp/repo_center/doris_master/doris/be/src/service/internal_service.cpp:1673:40 apache#6 0x556b407f52bb in doris::PInternalService::transmit_block(google::protobuf::RpcController*, doris::PTransmitDataParams const*, doris::PTransmitDataResult*, google::protobuf::Closure*) /home/zcp/repo_center/doris_master/doris/be/src/service/internal_service.cpp:1610:9 apache#7 0x556b43fceba2 in doris::PBackendService::CallMethod(google::protobuf::MethodDescriptor const*, google::protobuf::RpcController*, google::protobuf::Message const*, google::protobuf::Message*, google::protobuf::Closure*) /home/zcp/repo_center/doris_master/doris/gensrc/build/gen_cpp/internal_service.pb.cc:49452:7 apache#8 0x556b6736273e in brpc::policy::ProcessRpcRequest(brpc::InputMessageBase*) (/mnt/hdd01/selectdb-cloud-chaos/cluster0/be/lib/doris_be+0x770bd73e) apache#9 0x556b67357426 in brpc::ProcessInputMessage(void*) (/mnt/hdd01/selectdb-cloud-chaos/cluster0/be/lib/doris_be+0x770b2426) apache#10 0x556b67357f20 in brpc::InputMessenger::InputMessageClosure::~InputMessageClosure() (/mnt/hdd01/selectdb-cloud-chaos/cluster0/be/lib/doris_be+0x770b2f20) apache#11 0x556b673588dd in brpc::InputMessenger::OnNewMessages(brpc::Socket*) (/mnt/hdd01/selectdb-cloud-chaos/cluster0/be/lib/doris_be+0x770b38dd) apache#12 0x556b674a0adc in brpc::Socket::ProcessEvent(void*) (/mnt/hdd01/selectdb-cloud-chaos/cluster0/be/lib/doris_be+0x771fbadc) apache#13 0x556b672e0f76 in bthread::TaskGroup::task_runner(long) (/mnt/hdd01/selectdb-cloud-chaos/cluster0/be/lib/doris_be+0x7703bf76) apache#14 0x556b672cbbe0 in bthread_make_fcontext (/mnt/hdd01/selectdb-cloud-chaos/cluster0/be/lib/doris_be+0x77026be0) ``` ``` Note: The done pointer will be saved in add_block and may be called in another thread via done->Run(). For example, when blocks_size == 1, the process is as follows: transmit_block (i=0) └─> recvr->add_block(..., done, ...) // Pass done └─> SenderQueue::add_block └─> _pending_closures.push(done) // done is saved get_batch() [another thread] └─> closure_pair.first->Run() //⚠️ done->Run() is called └─> brpc releases request and response transmit_block (i=1) [original thread continues] └─> request->blocks_size() //⚠️ request has already been released! At this point, a use-after-free issue occurs. TODO: We should consider refactoring this part because add_block may release the request. We should not access the request after calling add_block. ``` apache#50113
optimize push down project, this can reduce the scan bytes and shuffle bytes by prune nested column. apache#57204 related the sql: ```sql select coalecse(struct_element(t1.s, 'city'), 'beijing') from t1 join t2 on t1.id = t2.id ``` original plan: ``` Project(coalecse(struct_element(t1.s, 'city'), 'beijing')) | Join(t1.id=t2.id) / \ Project(t1.id, t1.s) Project(t2.id) | | Scan(t1) Scan(t2) ``` optimize plan: ``` Project(coalecse(slot#3, 'beijing')) | Join(t1.id=t2.id) / \ Project(t1.id, struct_element(t1.s, 'city')#3) Project(t2.id) | | Scan(t1) Scan(t2) ```
…shard_ptr not initialized (apache#58751) ``` #0 0x000055f11470ff29 in std::__shared_ptr<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2>::get (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1673 #1 std::__shared_ptr_access<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2, false, false>::_M_get (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1370 #2 std::__shared_ptr_access<doris::TabletMeta, (__gnu_cxx::_Lock_policy)2, false, false>::operator-> (this=0x148) at /home/work/env/ldb_toolchain_master/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/shared_ptr_base.h:1364 #3 doris::BaseTablet::tablet_id (this=0x0) at /home/work/doris/be/src/olap/base_tablet.h:73 apache#4 doris::segment_v2::SegmentWriter::finalize (this=0x7fa160bde000, segment_file_size=0x7fa564dcc668, index_size=0x7fa564dcc5b8) at /home/work/doris/be/src/olap/rowset/segment_v2/segment_writer.cpp:1044 apache#5 0x000055f11442578c in doris::SegmentFlusher::_flush_segment_writer (this=0x7fa160b8a420, writer=..., flush_size=flush_size@entry=0x0) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:304 apache#6 0x000055f114426c4e in doris::SegmentFlusher::Writer::flush (this=<optimized out>) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:376 apache#7 doris::SegmentCreator::flush (this=0x7fa160b8a418) at /home/work/doris/be/src/olap/rowset/segment_creator.cpp:422 apache#8 0x000055f1143d4785 in doris::BaseBetaRowsetWriter::flush (this=<optimized out>) at /home/work/doris/be/src/olap/rowset/beta_rowset_writer.cpp:723 apache#9 0x000055f11436bf47 in doris::Merger::vmerge_rowsets (tablet=..., reader_type=<optimized out>, cur_tablet_schema=..., src_rowset_readers=..., dst_rowset_writer=0x7fa160b8a000, stats_output=0x7fa1df108298) at /home/work/doris/be/src/olap/merger.cpp:159 apache#10 0x000055f114345595 in doris::Compaction::merge_input_rowsets (this=this@entry=0x7fa1df108210) at /home/work/doris/be/src/olap/compaction.cpp:220 apache#11 0x000055f1143550d2 in doris::CloudCompactionMixin::execute_compact_impl (this=this@entry=0x7fa1df108210, permits=permits@entry=6) at /home/work/doris/be/src/olap/compaction.cpp:1491 apache#12 0x000055f114342bc1 in doris::CloudCompactionMixin::execute_compact (this=0x7fa1df108210) at /home/work/doris/be/src/olap/compaction.cpp:1620 apache#13 0x000055f11a22c6d8 in doris::CloudCumulativeCompaction::execute_compact (this=0x7fa1df108210) at /home/work/doris/be/src/cloud/cloud_cumulative_compaction.cpp:203 ```
…e#59098) ### What problem does this PR solve? Introduced by apache#58905 ==2037076==ERROR: AddressSanitizer: heap-use-after-free on address 0x7baaae908730 at pc 0x561b769a1fd0 bp 0x7b3caf4ebdf0 sp 0x7b3caf4ebde8 22:30:08 READ of size 1 at 0x7baaae908730 thread T12303 (rs_normal [work) 22:30:08 #0 0x561b769a1fcf in doris::(anonymous namespace)::string_compare(char const*, long, char const*, long, long) /root/doris/be/src/vec/common/string_ref.h:170:29 22:30:08 #1 0x561b769a1fcf in doris::StringRef::compare(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:259:30 22:30:08 #2 0x561b76f537cd in doris::StringRef::ge(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:282:52 22:30:08 #3 0x561b76f537cd in doris::StringRef::operator>=(doris::StringRef const&) const /root/doris/be/src/vec/common/string_ref.h:292:60 22:30:08 apache#4 0x561b76f537cd in bool doris::Compare::greater_equal<doris::StringRef>(doris::StringRef const&, doris::StringRef const&) /root/doris/be/src/common/compare.h:42:18 22:30:08 apache#5 0x561b76f537cd in doris::ComparisonPredicateBase<(doris::PrimitiveType)23, (doris::PredicateType)6>::camp_field(doris::vectorized::Field const&, doris::vectorized::Field const&) const /root/doris/be/src/olap/comparison_predicate.h:192:20 22:30:08 apache#6 0x561b76f4baa4 in doris::ComparisonPredicateBase<(doris::PrimitiveType)23, (doris::PredicateType)6>::evaluate_and(doris::vectorized::ParquetPredicate::ColumnStat*) const /root/doris/be/src/olap/comparison_predicate.h:207:26 22:30:08 apache#7 0x561b76765284 in doris::AndBlockColumnPredicate::evaluate_and(doris::vectorized::ParquetPredicate::ColumnStat*) const /root/doris/be/src/olap/block_column_predicate.h:251:42 22:30:08 apache#8 0x561b89acd735 in doris::vectorized::ParquetReader::_process_column_stat_filter(tparquet::RowGroup const&, std::vector<std::unique_ptr<doris::MutilColumnBlockPredicate, std::default_delete<doris::MutilColumnBlockPredicate> >, std::allocator<std::unique_ptr<doris::MutilColumnBlockPredicate, std::default_delete<doris::MutilColumnBlockPredicate> > > > const&, bool*, bool*, bool*) /root/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:1225:25 22:30:08 apache#9 0x561b89ac8dd7 in doris::vectorized::ParquetReader::_process_min_max_bloom_filter(doris::vectorized::RowGroupReader::RowGroupIndex const&, tparquet::RowGroup const&, std::vector<std::unique_ptr<doris::MutilColumnBlockPredicate, std::default_delete<doris::MutilColumnBlockPredicate> >, std::allocator<std::unique_ptr<doris::MutilColumnBlockPredicate, std::default_delete<doris::MutilColumnBlockPredicate> > > > const&, doris::segment_v2::RowRanges*) /root/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:1108:9 22:30:08 apache#10 0x561b89ac3e73 in doris::vectorized::ParquetReader::_next_row_group_reader() /root/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:718:9 22:30:08 apache#11 0x561b89ac008f in doris::vectorized::ParquetReader::get_next_block(doris::vectorized::Block*, unsigned long*, bool*) /root/doris/be/src/vec/exec/format/parquet/vparquet_reader.cpp:607:21 22:30:08 apache#12 0x561b8a07c6f7 in doris::vectorized::HiveReader::get_next_block_inner(doris::vectorized::Block*, unsigned long*, bool*) /root/doris/be/src/vec/exec/format/table/hive_reader.cpp:32:5 22:30:08 apache#13 0x561b89fee256 in doris::vectorized::TableFormatReader::get_next_block(doris::vectorized::Block*, unsigned long*, bool*) /root/doris/be/src/vec/exec/format/table/table_format_reader.h:81:16 22:30:08 apache#14 0x561b89f71b97 in doris::vectorized::FileScanner::_get_block_wrapped(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/vec/exec/scan/file_scanner.cpp:472:13 22:30:08 apache#15 0x561b89f7086f in doris::vectorized::FileScanner::_get_block_impl(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/vec/exec/scan/file_scanner.cpp:409:17 22:30:08 apache#16 0x561b8a19f86e in doris::vectorized::Scanner::get_block(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/vec/exec/scan/scanner.cpp:109:17 22:30:08 apache#17 0x561b8a19f0a6 in doris::vectorized::Scanner::get_block_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*) /root/doris/be/src/vec/exec/scan/scanner.cpp:85:16 22:30:08 apache#18 0x561b8a1ccd0f in doris::vectorized::ScannerScheduler::_scanner_scan(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>) /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:173:5 22:30:08 apache#19 0x561b8a1d6875 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:76:17 22:30:08 apache#20 0x561b8a1d6875 in doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()::operator()() const /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:75:27 22:30:08 apache#21 0x561b8a1d6875 in bool std::__invoke_impl<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>(std::__invoke_other, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63:14 22:30:08 apache#22 0x561b8a1d6875 in std::enable_if<is_invocable_r_v<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>, bool>::type std::__invoke_r<bool, doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&>(doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:116:9 22:30:08 apache#23 0x561b8a1d6875 in std::_Function_handler<bool (), doris::vectorized::ScannerScheduler::submit(std::shared_ptr<doris::vectorized::ScannerContext>, std::shared_ptr<doris::vectorized::ScanTask>)::$_0::operator()() const::'lambda'()>::_M_invoke(std::_Any_data const&) /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292:9 22:30:08 apache#24 0x561b8a1d5f07 in std::function<bool ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 22:30:08 apache#25 0x561b8a1d5f07 in doris::vectorized::ScannerSplitRunner::process_for(std::chrono::duration<long, std::ratio<1l, 1000000000l> >) /root/doris/be/src/vec/exec/scan/scanner_scheduler.cpp:407:25 22:30:08 apache#26 0x561b8a2c56d4 in doris::vectorized::PrioritizedSplitRunner::process() /root/doris/be/src/vec/exec/executor/time_sharing/prioritized_split_runner.cpp:103:35 22:30:08 apache#27 0x561b8a29045c in doris::vectorized::TimeSharingTaskExecutor::_dispatch_thread() /root/doris/be/src/vec/exec/executor/time_sharing/time_sharing_task_executor.cpp:570:77 22:30:08 apache#28 0x561b7b9fecb6 in std::function<void ()>::operator()() const /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593:9 22:30:08 apache#29 0x561b7b9fecb6 in doris::Thread::supervise_thread(void*) /root/doris/be/src/util/thread.cpp:460:5 22:30:08 apache#30 0x561b76044d26 in asan_thread_start(void*) (/mnt/ssd01/pipline/OpenSourceDoris/clusterEnv/P1/Cluster0/be/lib/doris_be+0x23962d26) 22:30:08 apache#31 0x7f4aaae68608 in start_thread /build/glibc-SzIz7B/glibc-2.31/nptl/pthread_create.c:477:8 22:30:08 apache#32 0x7f4aaad7b132 in __clone /build/glibc-SzIz7B/glibc-2.31/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95
…pache#55500) ### What problem does this PR solve? Related PR: apache#45937 Problem Summary: Fix the error case on ingestion load and the core in parquet reader. ==8898==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x62f0020603fc at pc 0x55f634e64ded bp 0x7fba0d03c410 sp 0x7fba0d03bbd8 READ of size 4 at 0x62f0020603fc thread T768 (PUSH-9699) #0 0x55f634e64dec in __asan_memcpy (/mnt/hdd01/ci/doris-deploy-branch-3.1-local/be/lib/doris_be+0x39a24dec) (BuildId: 9b04e7f7d3075dac) #1 0x55f634eca93f in std::char_traits::copy(char*, char const*, unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/char_traits.h:409:33 #2 0x55f634eca93f in std::__cxx11::basic_string, std::allocator>::_S_copy(char*, char const*, unsigned long) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:351:4 #3 0x55f634eca93f in std::__cxx11::basic_string, std::allocator>::_S_copy_chars(char*, char const*, char const*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:398:9 apache#4 0x55f634eca93f in void std::__cxx11::basic_string, std::allocator>::_M_construct(char const*, char const*, std::forward_iterator_tag) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.tcc:225:6 apache#5 0x55f654a4f74d in void std::__cxx11::basic_string, std::allocator>::_M_construct_aux(char const*, char const*, std::__false_type) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:247:11 apache#6 0x55f654a4f74d in void std::__cxx11::basic_string, std::allocator>::_M_construct(char const*, char const*) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:266:4 apache#7 0x55f654a4f74d in std::__cxx11::basic_string, std::allocator>::basic_string(char const*, unsigned long, std::allocator const&) /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:513:9 apache#8 0x55f654a4f74d in doris::vectorized::parse_thrift_footer(std::shared_ptr, doris::vectorized::FileMetaData**, unsigned long*, doris::io::IOContext*) /home/zcp/repo_center/doris_branch-3.1/doris/be/src/vec/exec/format/parquet/parquet_thrift_util.h:55:17
…pache#59920) ### What problem does this PR solve? When `BaseBetaRowsetWriter` is destructed(this may happen when the load is canceled) before the execution of the task submitted to thread pool, the task may encounter coredump due to use after free. ``` (gdb) bt #0 __GI___pthread_sigmask (how=2, newmask=<optimized out>, oldmask=0x0) at ./nptl/pthread_sigmask.c:43 #1 0x00007fa1d0c1171e in PosixSignals::chained_handler(int, siginfo*, void*) [clone .part.0] () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #2 0x00007fa1d0c12206 in JVM_handle_linux_signal () from /usr/lib/jvm/java-17-openjdk-amd64/lib/server/libjvm.so #3 <signal handler called> apache#4 doris::TUniqueId::TUniqueId (this=0x7f99955f2208, other51=...) at /home/zcp/repo_center/doris_branch-4.0/doris/gensrc/build/gen_cpp/Types_types.cpp:2571 apache#5 0x00005653d14008ca in doris::AttachTask::init (rc=..., this=<optimized out>) at /home/zcp/repo_center/doris_branch-4.0/doris/be/src/runtime/thread_context.cpp:29 apache#6 doris::AttachTask::AttachTask (this=<optimized out>, rc=...) at /home/zcp/repo_center/doris_branch-4.0/doris/be/src/runtime/thread_context.cpp:34 apache#7 0x00005653d0d05087 in doris::CalcDeleteBitmapToken::submit_func<doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0>(doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0&&)::{lambda()#1}::operator()() const (this=0x7f9cdf302500) at /home/zcp/repo_center/doris_branch-4.0/doris/be/src/olap/calc_delete_bitmap_executor.h:74 apache#8 std::__invoke_impl<void, doris::CalcDeleteBitmapToken::submit_func<doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0>(doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0&&)::{lambda()#1}&>(std::__invoke_other, doris::CalcDeleteBitmapToken::submit_func<doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0>(doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0&&)::{lambda()#1}&) (__f=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:63 apache#9 std::__invoke_r<void, doris::CalcDeleteBitmapToken::submit_func<doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0>(doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0&&)::{lambda()#1}&>(doris::CalcDeleteBitmapToken::submit_func<doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0>(doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0&&)::{lambda()#1}&) (__fn=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/invoke.h:113 apache#10 std::_Function_handler<void (), doris::CalcDeleteBitmapToken::submit_func<doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0>(doris::BaseBetaRowsetWriter::_generate_delete_bitmap(int)::$_0&&)::{lambda()#1}>::_M_invoke(std::_Any_data const&) ( __functor=...) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:292 apache#11 0x00005653d16392e5 in doris::ThreadPool::dispatch_thread (this=0x7fa120d9af00) at /home/zcp/repo_center/doris_branch-4.0/doris/be/src/util/threadpool.cpp:616 apache#12 0x00005653d162e38c in std::function<void ()>::operator()() const (this=0x7f99955f2208) at /usr/local/ldb-toolchain-v0.26/bin/../lib/gcc/x86_64-pc-linux-gnu/15/include/g++-v15/bits/std_function.h:593 apache#13 doris::Thread::supervise_thread (arg=0x7fa0c0049110) at /home/zcp/repo_center/doris_branch-4.0/doris/be/src/util/thread.cpp:460 apache#14 0x00007fa1cfcacac3 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442 apache#15 0x00007fa1cfd3e850 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 ``` ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
…lain verbose (apache#60308) ### What problem does this PR solve? Problem Summary: This PR enhances the output of EXPLAIN VERBOSE for File Scan nodes by adding the following metrics: `dataFileNum=xxx, deleteFileNum=xxx, deleteSplitNum=xxx` Especially useful for iceberg/paimon/hive acid These metrics provide more visibility into the underlying file and split layout, helping users better tune parameters and control query performance. Details: `dataFileNum` : The number of distinct data files that need to be read. This is not equivalent to the number of splits, since a single data file can be divided into multiple splits. `deleteFileNum` : The number of distinct delete files that need to be read. `deleteSplitNum` : Added because the relationship between data files and delete files is many-to-many: one data file may be associated with multiple delete files one delete file may apply to multiple data files Using deleteSplitNum / dataSplitNum, users can estimate the average number of delete splits that need to be read per data split. Example: ``` mysql> explain verbose select * from iceberg.format_v3.dv_test_1w; +-----------------------------------------------------------------------------------------------------------------------------------------------+ | Explain String(Nereids Planner) | +-----------------------------------------------------------------------------------------------------------------------------------------------+ | PLAN FRAGMENT 0 | | OUTPUT EXPRS: | | id[#0] | | grp[#1] | | value[#2] | | ts[#3] | | PARTITION: RANDOM | | | | HAS_COLO_PLAN_NODE: false | | | | VRESULT SINK | | MYSQL_PROTOCOL | | | | 0:VICEBERG_SCAN_NODE(32) | | table: iceberg.format_v3.dv_test_1w | | inputSplitNum=220, totalFileSize=720774, scanRanges=220 | | partition=0/0 | | backends: | | 1769590309070 | | s3://warehouse/wh/format_v3/dv_test_1w/data/00004-51-fc462f9a-d42a-404d-adfc-c8d2781c8d04-0-00001.parquet start: 4 length: 2672 | | s3://warehouse/wh/format_v3/dv_test_1w/data/00003-50-fc462f9a-d42a-404d-adfc-c8d2781c8d04-0-00001.parquet start: 4 length: 2852 | | s3://warehouse/wh/format_v3/dv_test_1w/data/00000-47-fc462f9a-d42a-404d-adfc-c8d2781c8d04-0-00001.parquet start: 4 length: 2894 | | ... other 216 files ... | | s3://warehouse/wh/format_v3/dv_test_1w/data/00001-48-fc462f9a-d42a-404d-adfc-c8d2781c8d04-0-00001.parquet start: 58397 length: 13894 | | dataFileNum=10, deleteFileNum=1 deleteSplitNum=220 | | cardinality=33334, numNodes=1 | | pushdown agg=NONE | | tuple ids: 0 | | | | Tuples: | | TupleDescriptor{id=0, tbl=dv_test_1w} | | SlotDescriptor{id=0, col=id, colUniqueId=1, type=bigint, nullable=true, isAutoIncrement=false, subColPath=null, virtualColumn=null} | | SlotDescriptor{id=1, col=grp, colUniqueId=2, type=int, nullable=true, isAutoIncrement=false, subColPath=null, virtualColumn=null} | | SlotDescriptor{id=2, col=value, colUniqueId=3, type=int, nullable=true, isAutoIncrement=false, subColPath=null, virtualColumn=null} | | SlotDescriptor{id=3, col=ts, colUniqueId=4, type=datetimev2(6), nullable=true, isAutoIncrement=false, subColPath=null, virtualColumn=null} | | | | | | | | | | ========== STATISTICS ========== | +-----------------------------------------------------------------------------------------------------------------------------------------------+ ```
…n_udf_support` is not set (apache#61196) When `enable_python_udf_support` is not set to true, a null pointer dereference occurs when invoking helper commands (`SHOW PYTHON VERSION/ PACKAGES IN`). ```text /home/zcp/repo_center/doris_master/doris/be/src/udf/python/python_env.cpp:305:38: runtime error: member call on null pointer of type 'doris::PythonEnvScanner' #0 0x559edede162c in doris::PythonVersionManager::env_infos_to_thrift() const /home/zcp/repo_center/doris_master/doris/be/src/udf/python/python_env.cpp:305:38 #1 0x559ede63acd4 in doris::BaseBackendService::get_python_envs(std::vector>&) /home/zcp/repo_center/doris_master/doris/be/src/service/backend_service.cpp:1316:47 #2 0x559edf99796a in doris::BackendServiceProcessor::process_get_python_envs(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) /home/zcp/repo_center/doris_master/doris/gensrc/build/gen_cpp/BackendService.cpp:7789:13 #3 0x559edf94a69c in doris::BackendServiceProcessor::dispatchCall(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__cxx11::basic_string, std::allocator> const&, int, void*) /home/zcp/repo_center/doris_master/doris/gensrc/build/gen_cpp/BackendService.cpp:6466:3 apache#4 0x559eac6bad06 in apache::thrift::TDispatchProcessor::process(std::shared_ptr, std::shared_ptr, void*) /home/zcp/repo_center/doris_master/doris/thirdparty/installed/include/thrift/TDispatchProcessor.h:121:12 apache#5 0x559ee47c071a in apache::thrift::server::TConnectedClient::run() (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x8a1c871a) apache#6 0x559ee47c1a45 in apache::thrift::server::TThreadedServer::TConnectedClientRunner::run() (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x8a1c9a45) apache#7 0x559ee47c557f in apache::thrift::concurrency::Thread::threadMain(std::shared_ptr) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x8a1cd57f) apache#8 0x559ee47c52a5 in void std::__invoke_impl), std::shared_ptr>(std::__invoke_other, void (*&&)(std::shared_ptr), std::shared_ptr&&) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x8a1cd2a5) apache#9 0x559ee47c521c in std::__invoke_result), std::shared_ptr>::type std::__invoke), std::shared_ptr>(void (*&&)(std::shared_ptr), std::shared_ptr&&) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x8a1cd21c) apache#10 0x559ee47c51f1 in void std::thread::_Invoker), std::shared_ptr>>::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x8a1cd1f1) apache#11 0x559ee47c51b4 in std::thread::_Invoker), std::shared_ptr>>::operator()() (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x8a1cd1b4) apache#12 0x559ee47c4fd8 in std::thread::_State_impl), std::shared_ptr>>>::_M_run() (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x8a1ccfd8) apache#13 0x559ef61b5f3f in execute_native_thread_routine archive64.c apache#14 0x559eac535d26 in asan_thread_start(void*) (/mnt/hdd01/ci/doris-deploy-master-local/be/lib/doris_be+0x51f3dd26) apache#15 0x7fcba62bfac2 in start_thread nptl/pthread_create.c:442:8 apache#16 0x7fcba635184f misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81 SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/zcp/repo_center/doris_master/doris/be/src/udf/python/python_env.cpp:305:38 ``` now: ```text Doris> show python packages in '3.19.18'; ERROR 1105 (HY000): errCode = 2, detailMessage = Failed to get python packages from any backend: [E42] Set 'python_venv_interpreter_paths' in be.conf to enable PythonUDF feature Doris> show python versions; ERROR 1105 (HY000): errCode = 2, detailMessage = Failed to get python envs from any backend: [E-236] Set 'python_venv_interpreter_paths' in be.conf to enable PythonUDF feature ```
… exit (apache#61040) ### What problem does this PR solve? The SyncPoint singleton was being destroyed during program exit while background bthread timers were still accessing it, causing heap-use-after-free. Changed get_instance() to return a never-destructed heap-allocated pointer instead of a stack-allocated static object to prevent destruction order issues. This is a standard pattern for singletons accessed by background threads. ``` 11:47:40 #0 0x5582a963153d in operator new(unsigned long) (/root/doris/cloud/ut_build_ASAN/test/recycler_test+0x122b53d) (BuildId: 1191affa24589f52) 11:47:40 #1 0x5582ab0aa52f in doris::SyncPoint::SyncPoint() /root/doris/common/cpp/sync_point.cpp:77:9 11:47:40 #2 0x5582ab0aa46b in doris::SyncPoint::get_instance() /root/doris/common/cpp/sync_point.cpp:73:20 11:47:40 #3 0x5582aae33ee8 in doris::cloud::memkv::Transaction::get(std::basic_string_view<char, std::char_traits<char>>, std::basic_string_view<char, std::char_traits<char>>, std::unique_ptr<doris::cloud::RangeGetIterator, std::default_delete<doris::cloud::RangeGetIterator>>*, bool, int) /root/doris/cloud/src/meta-store/mem_txn_kv.cpp:276:5 11:47:40 apache#4 0x5582a968f49f in doris::cloud::txn_get(doris::cloud::TxnKv*, std::basic_string_view<char, std::char_traits<char>>, std::basic_string_view<char, std::char_traits<char>>, std::unique_ptr<doris::cloud::RangeGetIterator, std::default_delete<doris::cloud::RangeGetIterator>>&) /root/doris/cloud/src/recycler/recycler.cpp:99:18 11:47:40 apache#5 0x5582a9657090 in doris::cloud::InstanceRecycler::scan_and_recycle(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::basic_string_view<char, std::char_traits<char>>, std::function<int (std::basic_string_view<char, std::char_traits<char>>, std::basic_string_view<char, std::char_traits<char>>)>, std::function<int ()>) /root/doris/cloud/src/recycler/recycler.cpp:2976:23 11:47:40 apache#6 0x5582a9686153 in doris::cloud::InstanceRecycler::recycle_rowsets() /root/doris/cloud/src/recycler/recycler.cpp:2587:15 11:47:40 apache#7 0x5582a96a8d47 in doris::cloud::RecyclerTest_recycle_empty_Test::TestBody() /root/doris/cloud/test/recycler_test.cpp:1136:5 ```
…free. (apache#61138) ### What problem does this PR solve? Related PR: apache/doris-thirdparty#383 Problem Summary: fix coredump: ``` #0 0x5561f1d46f12 in operator delete(void*, unsigned long) (/mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be+0x50ae4f12) #1 0x55622c94b454 in std::__new_allocator<std::shared_ptr<orc::ExpressionTree>>::deallocate(std::shared_ptr<orc::ExpressionTree>*, unsigned long) (/mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be+0x8b6e9454) #2 0x55622c94b41e in std::_Deque_base<std::shared_ptr<orc::ExpressionTree>, std::allocator<std::shared_ptr<orc::ExpressionTree>>>::_M_deallocate_node(std::shared_ptr<orc::ExpressionTree>*) (/mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be+0x8b6e941e) #3 0x55622c950681 in std::deque<std::shared_ptr<orc::ExpressionTree>, std::allocator<std::shared_ptr<orc::ExpressionTree>>>::_M_pop_front_aux() (/mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be+0x8b6ee681) apache#4 0x55622c94868e in std::deque<std::shared_ptr<orc::ExpressionTree>, std::allocator<std::shared_ptr<orc::ExpressionTree>>>::pop_front() (/mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be+0x8b6e668e) apache#5 0x55622c947d79 in orc::rewriteLeaves(std::shared_ptr<orc::ExpressionTree>, unsigned long*) SearchArgument.cc apache#6 0x55622c947809 in orc::SearchArgumentBuilderImpl::build() (/mnt/disk2/tengjianping/doris-master/output/be/lib/doris_be+0x8b6e5809) apache#7 0x5562195209a6 in doris::vectorized::OrcReader::_init_search_argument(std::vector<std::shared_ptr<doris::vectorized::VExpr>, std::allocator<std::shared_ptr<doris::vectorized::VExpr>>> const&) /mnt/disk2/tengjianping/doris-master/be/src/vec/exec/format/orc/vorc_reader.cpp:1072:27 apache#8 0x556219527175 in doris::vectorized::OrcReader::set_fill_columns(std::unordered_map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::tuple<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, doris::SlotDescriptor const*>, std::hash<std::__cxx11::basic_string<char, ```
```
==1472432==ERROR: AddressSanitizer: heap-use-after-free on address 0x7bc7535fba50 at pc 0x557ec5178739 bp 0x7b1743746d40 sp 0x7b1743746500
WRITE of size 104 at 0x7bc7535fba50 thread T2
#0 0x557ec5178738 in memset ~/workspace/llvm-project/compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:87:3
#1 0x7f17566e64fc in fdb_transaction_set_option (libfdb_c.so+0xc014fc) (BuildId: 854ec989d11d3c183aa4963a6d758cfc2b571b07)
#2 0x7f17566e2e47 in fdb_transaction_set_option (libfdb_c.so+0xbfde47) (BuildId: 854ec989d11d3c183aa4963a6d758cfc2b571b07)
#3 0x7f17566e4ec5 in fdb_transaction_set_option (libfdb_c.so+0xbffec5) (BuildId: 854ec989d11d3c183aa4963a6d758cfc2b571b07)
apache#4 0x7f17564f80ee in fdb_transaction_set_option (libfdb_c.so+0xa130ee) (BuildId: 854ec989d11d3c183aa4963a6d758cfc2b571b07)
apache#5 0x7f1755e55b71 in fdb_transaction_set_option (libfdb_c.so+0x370b71) (BuildId: 854ec989d11d3c183aa4963a6d758cfc2b571b07)
apache#6 0x7f1755e2b2e9 in fdb_run_network (libfdb_c.so+0x3462e9) (BuildId: 854ec989d11d3c183aa4963a6d758cfc2b571b07)
apache#7 0x557ec6595a2b in doris::cloud::fdb::Network::init()::::operator()() const ~/workspace/doris/cloud/ut_build_ASAN/../src/meta-store/txn_kv.cpp:468:28
apache#8 0x557ec65958e6 in void std::__invoke_impl<void, doris::cloud::fdb::Network::init()::>(std::__invoke_other, doris::cloud::fdb::Network::init()::&&) /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14
apache#9 0x557ec6595886 in std::__invoke_result<doris::cloud::fdb::Network::init()::>::type std::__invoke<doris::cloud::fdb::Network::init()::>(doris::cloud::fdb::Network::init()::&&) /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14
apache#10 0x557ec659584e in void std::thread::_Invoker<std::tuple<doris::cloud::fdb::Network::init()::>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:292:13
apache#11 0x557ec6595816 in std::thread::_Invoker<std::tuple<doris::cloud::fdb::Network::init()::>>::operator()() /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:299:11
apache#12 0x557ec659571a in std::thread::_State_impl<std::thread::_Invoker<std::tuple<doris::cloud::fdb::Network::init()::>>>::_M_run() /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:244:13
apache#13 0x557ec7756e43 in execute_native_thread_routine InstrProfilingUtil.c
apache#14 0x557ec50b1a07 in asan_thread_start(void*) ~/workspace/llvm-project/compiler-rt/lib/asan/asan_interceptors.cpp:239:43
apache#15 0x7f17545caaa3 in start_thread nptl/pthread_create.c:447:8
apache#16 0x7f1754657a63 in clone misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:100
```
Summary
function_search_testwith scenarios that comparesearchagainstmatch_any/match_allrow sets for OR across fields and OR with NOT on the same fieldTesting
https://chatgpt.com/codex/tasks/task_e_68da65558fe483268789ba22921fd836
Note
Analyze OR clauses for null-safety to decide when to mask/collect NULLs, and add targeted tests to validate behavior.
analyze_null_safetyto detectNOTand cross-field overlaps; introduceis_or_clause_null_safe.should_mask_null_valuesto mask only unsafeORtrees; preserve masking for others.collect_query_null_bitmap_internal(..., collect_nulls)to avoid reading/unioning NULL bitmaps when not needed; handlehas_null()asResult<bool>.function_search.h; include<unordered_set>/<vector>.TrackingIndexIteratorand new cases:Written by Cursor Bugbot for commit 67aa9bd. This will update automatically on new commits. Configure here.