Skip to content

[fix](pipelineX) fix DCHECK error when reuse _join_block in nested loop join#46493

Merged
liutang123 merged 1 commit into
apache:masterfrom
liutang123:fix-nlj-reuse
Jan 8, 2025
Merged

[fix](pipelineX) fix DCHECK error when reuse _join_block in nested loop join#46493
liutang123 merged 1 commit into
apache:masterfrom
liutang123:fix-nlj-reuse

Conversation

@liutang123

@liutang123 liutang123 commented Jan 7, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #46498

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@liutang123 liutang123 changed the title [fix](pipelineX) fix DCHECK error when reuse _join_block in nested lo… [fix](pipelineX) fix DCHECK error when reuse _join_block in nested loop join Jan 7, 2025
@liutang123

liutang123 commented Jan 7, 2025

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

doris-robot commented Jan 7, 2025

Copy link
Copy Markdown
TPC-H: Total hot run time: 32642 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f18a25738ce7cc9aa22c6cde67219c73ecbd3fac, data reload: false

------ Round 1 ----------------------------------
q1	17573	6174	6035	6035
q2	2050	314	164	164
q3	10459	1242	753	753
q4	10241	861	429	429
q5	7960	2191	1983	1983
q6	207	177	153	153
q7	904	762	610	610
q8	9222	1358	1180	1180
q9	5199	4912	4910	4910
q10	6780	2328	1850	1850
q11	492	276	250	250
q12	349	362	216	216
q13	17781	3692	3086	3086
q14	234	257	201	201
q15	553	513	497	497
q16	621	618	587	587
q17	580	845	322	322
q18	7006	6406	6398	6398
q19	2419	974	553	553
q20	301	308	185	185
q21	2775	2224	1970	1970
q22	356	337	310	310
Total cold run time: 104062 ms
Total hot run time: 32642 ms

----- Round 2, with runtime_filter_mode=off -----
q1	6292	6171	6245	6171
q2	236	326	230	230
q3	2218	2644	2360	2360
q4	1370	1821	1352	1352
q5	4359	4738	4835	4738
q6	188	173	141	141
q7	2102	1977	1783	1783
q8	2622	2802	2704	2704
q9	7315	7296	7273	7273
q10	3093	3330	2878	2878
q11	585	516	497	497
q12	643	740	616	616
q13	3496	3897	3266	3266
q14	302	307	276	276
q15	578	510	513	510
q16	641	702	647	647
q17	1251	1772	1259	1259
q18	7666	7614	7235	7235
q19	849	1085	1108	1085
q20	1993	2057	1938	1938
q21	5766	5260	4882	4882
q22	613	614	631	614
Total cold run time: 54178 ms
Total hot run time: 52455 ms

@doris-robot

doris-robot commented Jan 7, 2025

Copy link
Copy Markdown
TPC-DS: Total hot run time: 196204 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f18a25738ce7cc9aa22c6cde67219c73ecbd3fac, data reload: false

query1	1285	949	929	929
query2	6323	2265	2368	2265
query3	10951	4535	4642	4535
query4	33356	23607	23453	23453
query5	3775	614	458	458
query6	278	204	195	195
query7	3996	481	299	299
query8	302	240	216	216
query9	9574	2741	2756	2741
query10	430	301	244	244
query11	17991	15272	15106	15106
query12	158	103	105	103
query13	1562	546	410	410
query14	9766	7408	7039	7039
query15	253	212	198	198
query16	8202	618	460	460
query17	1581	776	592	592
query18	2131	412	319	319
query19	209	188	161	161
query20	123	122	114	114
query21	210	128	111	111
query22	4550	4542	4314	4314
query23	34444	33747	33556	33556
query24	6516	2352	2332	2332
query25	485	446	392	392
query26	774	271	155	155
query27	2056	458	327	327
query28	5361	2527	2479	2479
query29	633	552	422	422
query30	212	191	155	155
query31	1002	928	849	849
query32	81	59	57	57
query33	478	364	293	293
query34	772	867	512	512
query35	801	818	771	771
query36	1030	1051	983	983
query37	114	97	78	78
query38	4190	4194	4283	4194
query39	1558	1483	1475	1475
query40	206	126	102	102
query41	45	45	43	43
query42	124	105	101	101
query43	526	535	503	503
query44	1358	821	821	821
query45	189	179	167	167
query46	879	1070	670	670
query47	1992	2039	1958	1958
query48	391	429	326	326
query49	721	467	437	437
query50	635	679	395	395
query51	7413	7215	7141	7141
query52	101	99	94	94
query53	229	257	184	184
query54	472	498	396	396
query55	86	76	77	76
query56	250	254	246	246
query57	1240	1242	1181	1181
query58	242	227	222	222
query59	3256	3190	3233	3190
query60	337	261	251	251
query61	104	103	106	103
query62	837	830	764	764
query63	230	195	196	195
query64	3433	1008	645	645
query65	3343	3230	3305	3230
query66	840	418	319	319
query67	16331	15699	15482	15482
query68	8902	687	535	535
query69	481	292	238	238
query70	1235	1161	1167	1161
query71	443	279	251	251
query72	6246	3796	3635	3635
query73	656	765	357	357
query74	10098	9290	8932	8932
query75	4357	3169	2642	2642
query76	4000	1138	765	765
query77	769	352	285	285
query78	10063	10294	9403	9403
query79	5550	804	602	602
query80	700	515	442	442
query81	477	273	233	233
query82	330	147	119	119
query83	197	165	154	154
query84	279	95	70	70
query85	736	367	297	297
query86	361	329	296	296
query87	4551	4612	4497	4497
query88	3865	2258	2258	2258
query89	438	338	304	304
query90	2127	186	180	180
query91	130	131	111	111
query92	65	56	51	51
query93	3589	880	526	526
query94	657	395	277	277
query95	334	271	290	271
query96	488	606	289	289
query97	2879	2961	2859	2859
query98	216	202	195	195
query99	1663	1563	1435	1435
Total cold run time: 300205 ms
Total hot run time: 196204 ms

@doris-robot

doris-robot commented Jan 7, 2025

Copy link
Copy Markdown
ClickBench: Total hot run time: 31.4 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f18a25738ce7cc9aa22c6cde67219c73ecbd3fac, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.24	0.07	0.07
query4	1.60	0.11	0.11
query5	0.42	0.41	0.42
query6	1.15	0.64	0.64
query7	0.02	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.51	0.51
query10	0.55	0.57	0.54
query11	0.14	0.11	0.11
query12	0.14	0.10	0.11
query13	0.59	0.61	0.58
query14	2.76	2.90	2.77
query15	0.90	0.82	0.81
query16	0.38	0.37	0.38
query17	1.03	1.02	0.97
query18	0.23	0.21	0.20
query19	1.93	1.80	2.02
query20	0.01	0.01	0.01
query21	15.36	0.89	0.58
query22	0.75	0.86	0.77
query23	15.13	1.43	0.55
query24	2.77	1.11	2.89
query25	0.14	0.13	0.16
query26	0.33	0.15	0.13
query27	0.07	0.05	0.06
query28	14.07	1.47	1.04
query29	12.59	3.96	3.28
query30	0.25	0.08	0.05
query31	2.84	0.58	0.38
query32	3.23	0.54	0.45
query33	3.06	3.06	3.10
query34	16.70	5.11	4.47
query35	4.49	4.46	4.45
query36	0.64	0.48	0.48
query37	0.10	0.06	0.06
query38	0.04	0.03	0.03
query39	0.04	0.03	0.02
query40	0.17	0.12	0.13
query41	0.08	0.03	0.02
query42	0.03	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.72 s
Total hot run time: 31.4 s

@doris-robot

doris-robot commented Jan 7, 2025

Copy link
Copy Markdown

TeamCity be ut coverage result:
Function Coverage: 38.89% (10129/26048)
Line Coverage: 29.93% (85724/286401)
Region Coverage: 29.04% (43750/150672)
Branch Coverage: 25.57% (22324/87320)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f18a25738ce7cc9aa22c6cde67219c73ecbd3fac_f18a25738ce7cc9aa22c6cde67219c73ecbd3fac/report/index.html

@Gabriel39 Gabriel39 requested a review from zhangstar333 January 7, 2025 07:48
@zhangstar333

zhangstar333 commented Jan 7, 2025

Copy link
Copy Markdown
Contributor

could reproduce this coredump at master? @liutang123

@liutang123 liutang123 mentioned this pull request Jan 7, 2025
Merged
16 tasks
@liutang123

liutang123 commented Jan 7, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

could reproduce this coredump at master? @liutang123

Old planner is disabled in master. So, I can not reproduce it in master.
But, I can submit a testcase in 2.1 after this pr merged.
In addition, I test this code in my 2.1 cluster and it works well.

lide-reed
lide-reed approved these changes Jan 8, 2025
View reviewed changes

@lide-reed lide-reed left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions

github-actions Bot commented Jan 8, 2025

Copy link
Copy Markdown
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions Bot added approved Indicates a PR has been approved by one committer. reviewed labels Jan 8, 2025
@github-actions

github-actions Bot commented Jan 8, 2025

Copy link
Copy Markdown
Contributor

PR approved by anyone and no changes requested.

@liutang123 liutang123 merged commit b26aabe into apache:master Jan 8, 2025
liutang123 pushed a commit to liutang123/doris that referenced this pull request Jan 13, 2025
[fix](pipelineX) bp apache#46493 and add some regresstion tests
d87e149
liutang123 pushed a commit to liutang123/doris that referenced this pull request Jan 13, 2025
[fix](pipelineX) bp apache#46493 and add some regresstion tests
7f8187d
@liutang123 liutang123 mentioned this pull request Jan 13, 2025
Merged
liutang123 pushed a commit to liutang123/doris that referenced this pull request Jan 13, 2025
@liutang123
[fix](pipelineX) bp apache#46493 and add some regresstion tests
1afd953
liutang123 added a commit that referenced this pull request Jan 14, 2025
@liutang123
[fix](pipelineX) bp #46493 and add some regresstion tests (#46865)
a12f4d7 
### What problem does this PR solve?

bp #46493
Add some regresstion tests for #46493 and #46541

Co-authored-by: liulijia <liulijia1029@gmail.com>
github-actions Bot pushed a commit that referenced this pull request May 21, 2025
@liutang123
[fix](pipelineX) fix DCHECK error when reuse _join_block in nested lo…
df5fbda 
…op join (#46493)

Related Issue: #46498

Co-authored-by: liutang123 <liulijia@gmail.com>
@github-actions github-actions Bot mentioned this pull request May 21, 2025
Merged
dataroaring pushed a commit that referenced this pull request Jun 11, 2025
@github-actions @liutang123
branch-3.0: [fix](pipelineX) fix DCHECK error when reuse _join_block …
67b539c 
…in nested loop join #46493 (#51128)

Cherry-picked from #46493

Co-authored-by: Lijia Liu <liutang123@yeah.net>
Co-authored-by: liutang123 <liulijia@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lide-reed lide-reed lide-reed approved these changes
@cambyzju cambyzju cambyzju approved these changes
@zhangstar333 zhangstar333 Awaiting requested review from zhangstar333

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.8-merged dev/3.0.7-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@liutang123 @doris-robot @zhangstar333 @lide-reed @cambyzju @wm1581066 @dataroaring @morrySnow