Skip to content

branch-3.1: [fix](load_stream) close brpc stream after load stream is closed #56120#56287

Merged
morrySnow merged 1 commit into
apache:branch-3.1from
dataroaring:PR_56120
Sep 25, 2025
Merged

branch-3.1: [fix](load_stream) close brpc stream after load stream is closed #56120#56287
morrySnow merged 1 commit into
apache:branch-3.1from
dataroaring:PR_56120

Conversation

@dataroaring

@dataroaring dataroaring commented Sep 22, 2025

Copy link
Copy Markdown
Contributor

picked from #56120

…che#56120)

pick apache#56120.

Otherwise, auto partition on multi bes may lead to segment num mismatch
problem.

Co-authored-by: Yongqiang YANG <yangyogqiang@selectdb.com>
Co-authored-by: Xin Liao <liaoxinbit@126.com>
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring

Copy link
Copy Markdown
Contributor Author

run buildall

@doris-robot

Copy link
Copy Markdown

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 82.33% (1235/1500)
Line Coverage 66.19% (22228/33582)
Region Coverage 67.60% (11147/16490)
Branch Coverage 57.21% (5891/10298)

@doris-robot

Copy link
Copy Markdown
TPC-H: Total hot run time: 32740 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 777eeaa3958c636a9df83c7ec1c1f6faf3afb57b, data reload: false

------ Round 1 ----------------------------------
q1	17820	5535	5466	5466
q2	2038	406	281	281
q3	12362	1286	727	727
q4	10244	889	458	458
q5	8359	2431	2155	2155
q6	186	164	134	134
q7	913	749	626	626
q8	9351	1473	1198	1198
q9	5193	4973	4879	4879
q10	6779	2276	1844	1844
q11	492	277	261	261
q12	335	351	211	211
q13	17770	3589	2995	2995
q14	217	231	213	213
q15	530	451	461	451
q16	425	416	395	395
q17	622	870	374	374
q18	7136	6367	6553	6367
q19	1854	949	536	536
q20	325	335	211	211
q21	2810	2191	1954	1954
q22	1051	1030	1004	1004
Total cold run time: 106812 ms
Total hot run time: 32740 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5566	5491	5494	5491
q2	232	324	234	234
q3	2244	2685	2314	2314
q4	1354	1791	1337	1337
q5	4408	4988	5004	4988
q6	168	169	125	125
q7	2042	1924	1810	1810
q8	2648	2813	2754	2754
q9	7263	7253	7254	7253
q10	3017	3318	2673	2673
q11	570	528	499	499
q12	641	792	606	606
q13	3375	3780	3154	3154
q14	278	303	271	271
q15	519	468	458	458
q16	439	501	437	437
q17	1218	1725	1245	1245
q18	7600	7583	7311	7311
q19	807	1136	1071	1071
q20	2006	2077	1873	1873
q21	5436	4977	4653	4653
q22	1045	1092	985	985
Total cold run time: 52876 ms
Total hot run time: 51542 ms

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@doris-robot

Copy link
Copy Markdown
TPC-DS: Total hot run time: 192771 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 777eeaa3958c636a9df83c7ec1c1f6faf3afb57b, data reload: false

query1	951	411	401	401
query2	6233	2018	1912	1912
query3	8689	200	198	198
query4	33726	24016	23609	23609
query5	3685	586	454	454
query6	285	198	189	189
query7	4191	486	325	325
query8	312	241	236	236
query9	9223	2621	2600	2600
query10	465	337	263	263
query11	18141	15369	15169	15169
query12	159	104	106	104
query13	1564	553	416	416
query14	10125	7131	6970	6970
query15	257	187	177	177
query16	8102	684	550	550
query17	1585	788	593	593
query18	2158	425	334	334
query19	243	197	167	167
query20	123	118	119	118
query21	206	129	111	111
query22	4493	4680	4535	4535
query23	35368	33997	34017	33997
query24	7425	2698	2652	2652
query25	543	521	436	436
query26	1159	287	171	171
query27	2240	489	385	385
query28	5491	2281	2245	2245
query29	783	607	476	476
query30	233	198	176	176
query31	1055	941	806	806
query32	88	63	65	63
query33	517	390	343	343
query34	762	867	522	522
query35	782	815	725	725
query36	1024	1074	1008	1008
query37	115	94	70	70
query38	4044	4036	4019	4019
query39	1525	1633	1511	1511
query40	225	129	109	109
query41	49	47	46	46
query42	130	108	105	105
query43	487	521	470	470
query44	1340	825	828	825
query45	183	178	169	169
query46	888	1058	688	688
query47	1957	2047	1953	1953
query48	416	419	362	362
query49	779	491	398	398
query50	667	710	435	435
query51	7377	7254	7233	7233
query52	102	98	93	93
query53	231	261	186	186
query54	558	548	479	479
query55	83	81	80	80
query56	273	271	258	258
query57	1287	1291	1192	1192
query58	255	224	236	224
query59	3048	3227	3119	3119
query60	285	292	268	268
query61	126	109	110	109
query62	798	768	683	683
query63	231	196	194	194
query64	4447	1002	666	666
query65	3357	3309	3335	3309
query66	980	409	308	308
query67	16477	16069	15652	15652
query68	7468	828	566	566
query69	491	308	266	266
query70	1193	1153	1100	1100
query71	374	295	262	262
query72	5683	3773	3841	3773
query73	641	743	354	354
query74	10595	9177	9018	9018
query75	3213	3134	2684	2684
query76	3244	1170	752	752
query77	520	393	283	283
query78	10366	10458	9543	9543
query79	3113	882	599	599
query80	634	527	431	431
query81	503	257	223	223
query82	554	118	86	86
query83	157	168	145	145
query84	242	103	89	89
query85	777	357	298	298
query86	342	310	295	295
query87	4342	4330	4237	4237
query88	4938	2420	2408	2408
query89	413	340	286	286
query90	1807	187	187	187
query91	140	146	108	108
query92	62	57	50	50
query93	1472	923	572	572
query94	681	412	299	299
query95	339	279	266	266
query96	488	607	279	279
query97	3192	3294	3152	3152
query98	228	210	200	200
query99	1572	1404	1311	1311
Total cold run time: 294890 ms
Total hot run time: 192771 ms

@doris-robot

Copy link
Copy Markdown
ClickBench: Total hot run time: 29.01 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 777eeaa3958c636a9df83c7ec1c1f6faf3afb57b, data reload: false

query1	0.03	0.03	0.04
query2	0.08	0.04	0.04
query3	0.23	0.07	0.07
query4	1.62	0.08	0.08
query5	0.53	0.50	0.49
query6	1.13	0.74	0.74
query7	0.02	0.02	0.01
query8	0.06	0.04	0.04
query9	0.57	0.51	0.51
query10	0.56	0.56	0.56
query11	0.15	0.12	0.12
query12	0.15	0.12	0.12
query13	0.62	0.61	0.60
query14	0.78	0.79	0.78
query15	0.86	0.84	0.84
query16	0.38	0.36	0.38
query17	1.07	1.02	1.07
query18	0.19	0.19	0.20
query19	1.83	1.79	1.76
query20	0.02	0.01	0.01
query21	15.40	0.93	0.64
query22	0.76	0.75	0.67
query23	14.78	1.41	0.69
query24	2.25	0.37	0.21
query25	0.15	0.09	0.08
query26	0.28	0.18	0.17
query27	0.08	0.08	0.08
query28	13.42	1.20	0.56
query29	12.66	4.03	3.38
query30	0.25	0.08	0.06
query31	2.82	0.59	0.39
query32	3.22	0.56	0.48
query33	3.02	3.03	3.08
query34	16.62	5.18	4.55
query35	4.62	4.67	4.55
query36	0.63	0.48	0.47
query37	0.21	0.17	0.17
query38	0.16	0.16	0.17
query39	0.06	0.04	0.04
query40	0.16	0.14	0.13
query41	0.10	0.05	0.06
query42	0.06	0.05	0.04
query43	0.05	0.04	0.04
Total cold run time: 102.64 s
Total hot run time: 29.01 s

@doris-robot

Copy link
Copy Markdown

BE UT Coverage Report

Increment line coverage 56.00% (14/25) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 45.57% (12785/28053)
Line Coverage 36.41% (114055/313264)
Region Coverage 34.03% (65227/191650)
Branch Coverage 31.05% (34220/110212)

@hello-stephen

Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 80.00% (20/25) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 76.45% (21093/27590)
Line Coverage 69.80% (217970/312298)
Region Coverage 67.73% (130329/192427)
Branch Coverage 61.22% (67785/110726)

@morrySnow morrySnow changed the title [fix](load_stream) close brpc stream after load stream is closed (#56… branch-3.1: [fix](load_stream) close brpc stream after load stream is closed #56120 Sep 25, 2025
@morrySnow morrySnow merged commit a8522da into apache:branch-3.1 Sep 25, 2025
24 of 25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants