Skip to content

[Opt](cloud) Add segment prefetcher #59482

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
bobhan1 wants to merge 4 commits into
base: master
Choose a base branch
from
Open

[Opt](cloud) Add segment prefetcher #59482

bobhan1 wants to merge 4 commits into from
+1,156 −18

Conversation

@bobhan1
Copy link
Contributor

@bobhan1 bobhan1 commented Dec 30, 2025 •
edited
Loading

What problem does this PR solve?

Cold run test result

  • Environment: 1 FE + 2 BE(aliyun, ecs.i4g.16xlarge, 2.7 GHz Intel® Xeon® Scalable (Ice Lake) processors with high-performance local NVMe SSDs)
    • Network baseline bandwidth 32 Gbit/s on each machine
  • Duplicate table, 16 buckets, 6178831001 rows, data size is 396.837 GB
  • Cold run means query without any doris page, OS page cache or local disk file cache, all data will be fetched from S3.
  • Query: select count(user_id) from test_dup3 where env_id % 10 = 0 and user_id%10=0 and time % 10 =0 and event_id % 10 = 0 and length(domain) % 10 = 0 and length(path) % 10 = 0 and length(query)%10=0;
    • For this query, each BE will download 114GB data to local disk file cache from S3
    • 8 instance * 48 scanner = 384 scanner on each BE, and scanner thread pool size is set to 48
no prefetch prefetch_block_size=2
query time 2min47sec 39sec273ms
network bandwidth
(on one BE)		 slow fast

based on #59476

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Dec 30, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@bobhan1 bobhan1 force-pushed the segment-prefetcher branch 3 times, most recently from 097fafb to bb82c00 Compare December 31, 2025 07:50
@bobhan1 bobhan1 force-pushed the segment-prefetcher branch 3 times, most recently from 002d019 to 93885ad Compare January 7, 2026 02:24
@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 7, 2026

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 7, 2026

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.53% (1779/2237)
Line Coverage 64.72% (31481/48643)
Region Coverage 65.38% (15655/23943)
Branch Coverage 55.99% (8318/14856)

@dataroaring
Copy link
Contributor

dataroaring commented Jan 7, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 7, 2026

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.53% (1779/2237)
Line Coverage 64.74% (31490/48643)
Region Coverage 65.41% (15660/23943)
Branch Coverage 56.00% (8320/14856)

@doris-robot
Copy link

doris-robot commented Jan 8, 2026

BE UT Coverage Report

Increment line coverage 27.93% (205/734) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.03% (18997/35820)
Line Coverage 39.07% (175906/450188)
Region Coverage 33.62% (136271/405311)
Branch Coverage 34.58% (58772/169962)

@bobhan1 bobhan1 force-pushed the segment-prefetcher branch from e8cc3ca to ebb565f Compare January 12, 2026 02:43
@dataroaring
Copy link
Contributor

dataroaring commented Jan 12, 2026

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 12, 2026

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 79.57% (1784/2242)
Line Coverage 64.75% (31727/48997)
Region Coverage 65.44% (15783/24120)
Branch Coverage 56.01% (8384/14968)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 12, 2026

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

doris-robot commented Jan 12, 2026

TPC-H: Total hot run time: 31722 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ebb565fbdf359710e9e26ce8ccbe6cecd41c00ad, data reload: false

------ Round 1 ----------------------------------
q1	17675	4186	4054	4054
q2	2023	357	256	256
q3	10564	1305	720	720
q4	10331	817	310	310
q5	9683	2137	1834	1834
q6	225	173	138	138
q7	944	790	649	649
q8	9277	1400	1131	1131
q9	5140	4547	4654	4547
q10	6868	1830	1401	1401
q11	516	307	271	271
q12	753	745	581	581
q13	17838	3829	3079	3079
q14	287	298	272	272
q15	582	516	505	505
q16	704	704	633	633
q17	679	828	487	487
q18	6783	6388	7247	6388
q19	1153	1058	630	630
q20	407	381	261	261
q21	3259	2695	2536	2536
q22	1106	1129	1039	1039
Total cold run time: 106797 ms
Total hot run time: 31722 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4354	4327	4245	4245
q2	321	418	320	320
q3	2325	2830	2358	2358
q4	1465	1884	1564	1564
q5	4450	4345	4571	4345
q6	207	165	127	127
q7	2000	1926	1816	1816
q8	2567	2337	2412	2337
q9	7224	7246	7169	7169
q10	2511	2729	2249	2249
q11	529	452	428	428
q12	668	702	575	575
q13	3348	3817	3125	3125
q14	264	280	259	259
q15	530	492	487	487
q16	610	675	630	630
q17	1105	1289	1366	1289
q18	7534	7284	7245	7245
q19	856	802	819	802
q20	1908	1944	1795	1795
q21	4531	4334	4193	4193
q22	1065	993	958	958
Total cold run time: 50372 ms
Total hot run time: 48316 ms

@doris-robot
Copy link

doris-robot commented Jan 12, 2026

TPC-DS: Total hot run time: 172293 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ebb565fbdf359710e9e26ce8ccbe6cecd41c00ad, data reload: false

query5	4351	615	426	426
query6	324	224	210	210
query7	4238	456	262	262
query8	336	249	235	235
query9	8743	2685	2639	2639
query10	518	360	330	330
query11	15337	15166	14795	14795
query12	165	114	122	114
query13	1252	476	373	373
query14	6356	2942	2690	2690
query14_1	2591	2555	2557	2555
query15	200	186	167	167
query16	1001	477	439	439
query17	1079	681	575	575
query18	2526	439	343	343
query19	238	225	196	196
query20	123	121	114	114
query21	221	150	129	129
query22	4058	4067	3827	3827
query23	15928	15631	15471	15471
query23_1	15321	15460	15398	15398
query24	7413	1538	1171	1171
query24_1	1177	1161	1178	1161
query25	551	478	424	424
query26	1241	263	158	158
query27	2786	452	295	295
query28	4567	2161	2151	2151
query29	795	520	421	421
query30	308	237	210	210
query31	803	632	568	568
query32	80	63	66	63
query33	526	347	280	280
query34	921	872	521	521
query35	718	750	663	663
query36	886	843	858	843
query37	129	91	78	78
query38	2737	2694	2550	2550
query39	763	740	746	740
query39_1	713	708	711	708
query40	215	127	111	111
query41	65	61	60	60
query42	102	101	99	99
query43	443	464	427	427
query44	1304	727	727	727
query45	186	179	173	173
query46	841	947	594	594
query47	1474	1484	1401	1401
query48	319	317	238	238
query49	604	408	323	323
query50	630	274	194	194
query51	3769	3744	3804	3744
query52	102	107	96	96
query53	289	321	287	287
query54	278	256	232	232
query55	76	73	70	70
query56	273	281	280	280
query57	993	1060	965	965
query58	261	253	238	238
query59	2180	2083	2055	2055
query60	312	303	281	281
query61	160	154	153	153
query62	411	363	311	311
query63	294	266	262	262
query64	4883	1273	991	991
query65	3693	3723	3736	3723
query66	1405	411	298	298
query67	14943	14760	15113	14760
query68	2873	1016	745	745
query69	438	356	301	301
query70	1019	939	929	929
query71	305	297	273	273
query72	5715	3529	3638	3529
query73	597	713	312	312
query74	8734	8752	8621	8621
query75	2799	2823	2482	2482
query76	2441	1046	640	640
query77	335	385	290	290
query78	9910	9893	9244	9244
query79	1028	897	608	608
query80	1454	585	477	477
query81	542	258	229	229
query82	881	139	108	108
query83	365	244	245	244
query84	256	114	105	105
query85	1155	513	446	446
query86	393	332	316	316
query87	2801	2901	2769	2769
query88	3217	2243	2267	2243
query89	378	352	332	332
query90	1880	153	151	151
query91	174	176	139	139
query92	69	65	63	63
query93	984	913	537	537
query94	644	324	290	290
query95	554	364	310	310
query96	586	452	204	204
query97	2371	2416	2287	2287
query98	220	201	197	197
query99	592	580	495	495
Total cold run time: 247061 ms
Total hot run time: 172293 ms

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 12, 2026

BE UT Coverage Report

Increment line coverage 27.93% (205/734) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.96% (18997/35869)
Line Coverage 39.04% (176071/451050)
Region Coverage 33.61% (136318/405614)
Branch Coverage 34.62% (58893/170128)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 13, 2026

BE Regression && UT Coverage Report

Increment line coverage 37.18% (274/737) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.21% (25381/35151)
Line Coverage 59.02% (265861/450464)
Region Coverage 53.91% (221108/410122)
Branch Coverage 55.54% (94917/170889)

@bobhan1 bobhan1 force-pushed the segment-prefetcher branch from ebb565f to f2efab2 Compare January 16, 2026 03:27
@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 16, 2026

run buildall

@bobhan1 bobhan1 marked this pull request as ready for review January 16, 2026 03:30
@bobhan1
Copy link
Contributor Author

bobhan1 commented Jan 16, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

TPC-H: Total hot run time: 31522 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f2efab26178bd89c271613b1c7e917122c13eadd, data reload: false

------ Round 1 ----------------------------------
q1	17634	4221	4060	4060
q2	2094	349	248	248
q3	10107	1317	709	709
q4	10217	873	315	315
q5	7635	2160	1850	1850
q6	229	169	137	137
q7	941	783	666	666
q8	9275	1384	1105	1105
q9	4973	4611	4652	4611
q10	6784	1794	1412	1412
q11	509	300	281	281
q12	724	722	584	584
q13	17787	3809	3087	3087
q14	296	306	281	281
q15	594	519	512	512
q16	702	693	642	642
q17	662	730	599	599
q18	6933	6490	6271	6271
q19	1307	975	627	627
q20	383	355	254	254
q21	3023	2467	2275	2275
q22	1038	1058	996	996
Total cold run time: 103847 ms
Total hot run time: 31522 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4161	4095	4048	4048
q2	323	391	322	322
q3	2119	2645	2228	2228
q4	1309	1750	1322	1322
q5	4193	3992	3997	3992
q6	220	172	133	133
q7	1889	1808	1937	1808
q8	2651	2700	2592	2592
q9	7316	7229	7189	7189
q10	2482	2720	2301	2301
q11	555	478	457	457
q12	734	719	669	669
q13	3696	4103	3603	3603
q14	379	321	280	280
q15	551	508	490	490
q16	624	669	630	630
q17	1162	1356	1561	1356
q18	8175	7723	7866	7723
q19	871	892	834	834
q20	1974	2060	1925	1925
q21	5170	4425	4222	4222
q22	1075	1016	983	983
Total cold run time: 51629 ms
Total hot run time: 49107 ms

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

TPC-DS: Total hot run time: 175281 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f2efab26178bd89c271613b1c7e917122c13eadd, data reload: false

query5	4429	630	489	489
query6	336	221	225	221
query7	4217	467	255	255
query8	344	250	243	243
query9	8702	2939	2945	2939
query10	466	367	327	327
query11	15286	15124	14937	14937
query12	177	119	113	113
query13	1252	481	385	385
query14	6048	3100	2832	2832
query14_1	2806	2711	2724	2711
query15	204	193	176	176
query16	997	494	470	470
query17	1124	683	579	579
query18	2442	441	336	336
query19	223	233	201	201
query20	124	120	114	114
query21	219	138	118	118
query22	3831	4024	3971	3971
query23	15962	15580	15346	15346
query23_1	15541	15567	15554	15554
query24	7171	1573	1161	1161
query24_1	1172	1191	1209	1191
query25	545	458	420	420
query26	1255	268	149	149
query27	2767	449	281	281
query28	4574	2176	2173	2173
query29	770	544	443	443
query30	312	238	208	208
query31	820	625	571	571
query32	87	77	85	77
query33	543	361	330	330
query34	910	893	536	536
query35	752	780	708	708
query36	877	869	882	869
query37	129	95	84	84
query38	2776	2724	2654	2654
query39	784	751	732	732
query39_1	734	733	707	707
query40	225	142	125	125
query41	112	63	63	63
query42	107	100	103	100
query43	438	472	472	472
query44	1315	739	735	735
query45	185	186	173	173
query46	819	955	598	598
query47	1411	1509	1391	1391
query48	310	312	235	235
query49	600	434	347	347
query50	617	273	204	204
query51	3817	3770	3825	3770
query52	108	115	102	102
query53	291	322	286	286
query54	290	263	267	263
query55	87	81	77	77
query56	303	311	293	293
query57	1032	1010	962	962
query58	267	268	258	258
query59	1978	2194	2140	2140
query60	334	332	317	317
query61	157	151	144	144
query62	396	351	310	310
query63	305	268	273	268
query64	4916	1239	944	944
query65	3803	3798	3827	3798
query66	1455	426	321	321
query67	15518	15605	15387	15387
query68	2436	1078	752	752
query69	457	362	320	320
query70	977	958	963	958
query71	313	308	298	298
query72	5287	3402	3460	3402
query73	585	737	309	309
query74	8697	8734	8555	8555
query75	2767	2826	2500	2500
query76	2267	1076	653	653
query77	364	386	314	314
query78	9783	9956	9250	9250
query79	1105	941	575	575
query80	1309	597	490	490
query81	529	261	231	231
query82	1328	147	110	110
query83	340	253	237	237
query84	247	108	92	92
query85	1027	514	431	431
query86	408	296	323	296
query87	2884	2902	2815	2815
query88	3484	2590	2568	2568
query89	388	344	315	315
query90	1877	180	164	164
query91	171	163	136	136
query92	81	76	72	72
query93	927	906	537	537
query94	610	317	311	311
query95	599	391	323	323
query96	643	510	231	231
query97	2377	2368	2354	2354
query98	207	204	204	204
query99	624	594	529	529
Total cold run time: 246850 ms
Total hot run time: 175281 ms

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

ClickBench: Total hot run time: 27.17 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f2efab26178bd89c271613b1c7e917122c13eadd, data reload: false

query1	0.06	0.04	0.04
query2	0.10	0.05	0.04
query3	0.26	0.09	0.08
query4	1.60	0.12	0.12
query5	0.27	0.26	0.24
query6	1.15	0.67	0.66
query7	0.03	0.03	0.02
query8	0.05	0.04	0.04
query9	0.58	0.51	0.49
query10	0.56	0.55	0.53
query11	0.14	0.10	0.10
query12	0.14	0.11	0.11
query13	0.61	0.59	0.59
query14	0.96	0.95	0.94
query15	0.79	0.77	0.79
query16	0.39	0.40	0.42
query17	1.06	1.04	1.07
query18	0.24	0.21	0.22
query19	1.87	1.89	1.91
query20	0.02	0.02	0.01
query21	15.44	0.25	0.14
query22	5.06	0.05	0.05
query23	15.74	0.29	0.11
query24	1.40	0.76	0.53
query25	0.10	0.08	0.09
query26	0.14	0.14	0.13
query27	0.08	0.06	0.05
query28	5.06	1.06	0.89
query29	12.53	3.95	3.12
query30	0.27	0.14	0.12
query31	2.81	0.65	0.41
query32	3.27	0.56	0.46
query33	3.05	3.12	3.08
query34	16.08	5.12	4.42
query35	4.43	4.45	4.42
query36	0.65	0.51	0.49
query37	0.10	0.06	0.07
query38	0.07	0.05	0.04
query39	0.05	0.03	0.03
query40	0.17	0.13	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.55 s
Total hot run time: 27.17 s

@doris-robot
Copy link

doris-robot commented Jan 16, 2026

BE UT Coverage Report

Increment line coverage 17.88% (108/604) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.96% (19020/35916)
Line Coverage 39.01% (176213/451655)
Region Coverage 33.60% (136539/406337)
Branch Coverage 34.63% (59015/170397)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 16, 2026

BE Regression && UT Coverage Report

Increment line coverage 26.61% (161/605) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.11% (25376/35193)
Line Coverage 58.85% (265471/451064)
Region Coverage 53.48% (219697/410840)
Branch Coverage 55.21% (94502/171158)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gavinchou gavinchou Awaiting requested review from gavinchou gavinchou is a code owner

@dataroaring dataroaring Awaiting requested review from dataroaring dataroaring is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

dev/4.0.x

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@bobhan1 @Thearas @hello-stephen @dataroaring @doris-robot