Skip to content

Conversation

@feiniaofeiafei
Copy link
Contributor

@feiniaofeiafei feiniaofeiafei commented Dec 17, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:
This PR add a rewrite rule:

/**
 * This rule will rewrite grouping sets. eg:
 *      select a, b, c, d, e sum(f) from t group by rollup(a, b, c, d, e);
 * rewrite to:
 *    with cte1 as (select a, b, c, d, e, sum(f) x from t group by rollup(a, b, c, d, e))
 *      select * fom cte1
 *      union all
 *      select a, b, c, d, null, sum(x) x from t group by rollup(a, b, c, d)
 *
 * LogicalAggregate(gby: a,b,c,d,e,grouping_id output:a,b,c,d,e,grouping_id,sum(f))
 *   +--LogicalRepeat(grouping sets: (a,b,c,d,e),(a,b,c,d),(a,b,c),(a,b),(a),())
 * ->
 * LogicalCTEAnchor
 *   +--LogicalCTEProducer(cte)
 *     +--LogicalAggregate(gby: a,b,c,d,e; aggFunc: sum(f) as x)
 *   +--LogicalUnionAll
 *     +--LogicalProject(a,b,c,d, null as e, sum(x))
 *       +--LogicalAggregate(gby:a,b,c,d,grouping_id; aggFunc: sum(x))
 *         +--LogicalRepeat(grouping sets: (a,b,c,d),(a,b,c),(a,b),(a),())
 *           +--LogicalCTEConsumer(aggregateConsumer)
 *     +--LogicalCTEConsumer(directConsumer)
 */

The scenario where performance optimization is achieved is when pre-aggregation significantly reduces the amount of data. This rewrite reduces the number of rows generated by the repeat operator, resulting in improved performance.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 35969 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 994180846dd28e9bdffe341fcf517654fa2351fd, data reload: false

------ Round 1 ----------------------------------
q1	17685	4196	4048	4048
q2	2042	350	231	231
q3	10165	1328	718	718
q4	10218	845	306	306
q5	7504	2148	1906	1906
q6	191	172	137	137
q7	1025	855	706	706
q8	9377	1407	1141	1141
q9	7010	5331	5303	5303
q10	6862	2395	1950	1950
q11	526	338	300	300
q12	702	721	604	604
q13	17810	3700	3045	3045
q14	282	289	286	286
q15	601	532	524	524
q16	690	675	633	633
q17	690	825	533	533
q18	7447	7422	7877	7422
q19	1379	1001	662	662
q20	427	369	260	260
q21	4690	4291	4195	4195
q22	1162	1080	1059	1059
Total cold run time: 108485 ms
Total hot run time: 35969 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4316	4317	4335	4317
q2	334	424	312	312
q3	2510	3103	2463	2463
q4	1472	1897	1425	1425
q5	4524	4457	4498	4457
q6	213	168	123	123
q7	2007	2008	1770	1770
q8	2757	2572	2519	2519
q9	7423	7771	7345	7345
q10	2910	3122	2668	2668
q11	576	493	485	485
q12	609	725	594	594
q13	3365	3663	3032	3032
q14	280	286	252	252
q15	561	505	489	489
q16	611	642	600	600
q17	1117	1410	1385	1385
q18	7415	7028	7168	7028
q19	831	791	830	791
q20	1906	1965	1802	1802
q21	4686	4303	4206	4206
q22	1087	1020	975	975
Total cold run time: 51510 ms
Total hot run time: 49038 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 175590 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 994180846dd28e9bdffe341fcf517654fa2351fd, data reload: false

query5	5040	601	449	449
query6	346	252	214	214
query7	4232	464	273	273
query8	300	255	251	251
query9	8812	2538	2558	2538
query10	549	361	339	339
query11	15412	15204	14573	14573
query12	196	143	113	113
query13	1260	488	389	389
query14	
query14_1	6607	2877	2615	2615
query15	214	209	182	182
query16	912	497	459	459
query17	1091	695	608	608
query18	2689	441	332	332
query19	234	236	209	209
query20	127	116	117	116
query21	219	145	116	116
query22	4180	4039	3938	3938
query23	16545	16258	15879	15879
query23_1	16108	16092	16098	16092
query24	7332	1676	1239	1239
query24_1	1259	1249	1232	1232
query25	585	508	452	452
query26	1254	280	176	176
query27	2742	463	311	311
query28	4496	2137	2139	2137
query29	851	573	484	484
query30	324	246	211	211
query31	821	713	646	646
query32	82	76	73	73
query33	561	344	304	304
query34	910	901	544	544
query35	794	821	733	733
query36	879	903	828	828
query37	128	90	86	86
query38	2875	2898	2814	2814
query39	760	760	734	734
query39_1	707	706	699	699
query40	227	152	136	136
query41	74	73	70	70
query42	116	106	109	106
query43	406	433	408	408
query44	1323	753	748	748
query45	199	201	190	190
query46	895	988	620	620
query47	1700	1700	1634	1634
query48	328	356	255	255
query49	638	450	366	366
query50	670	300	242	242
query51	3820	3769	3826	3769
query52	110	110	99	99
query53	314	347	302	302
query54	305	273	261	261
query55	84	78	74	74
query56	324	307	303	303
query57	1171	1146	1075	1075
query58	281	263	266	263
query59	2437	2509	2379	2379
query60	323	365	292	292
query61	167	155	160	155
query62	696	678	617	617
query63	323	303	298	298
query64	5044	1287	1024	1024
query65	4054	3967	3978	3967
query66	1393	442	340	340
query67	15118	15269	15056	15056
query68	2414	1117	782	782
query69	433	365	307	307
query70	1062	1001	975	975
query71	316	312	280	280
query72	5411	4855	4782	4782
query73	501	541	303	303
query74	8890	8852	8588	8588
query75	3093	3185	2824	2824
query76	2723	1143	714	714
query77	346	388	296	296
query78	9494	9586	8844	8844
query79	2039	891	638	638
query80	1349	622	549	549
query81	552	270	245	245
query82	996	132	104	104
query83	364	262	234	234
query84	260	114	102	102
query85	963	522	467	467
query86	402	325	290	290
query87	3045	3044	2992	2992
query88	3332	2282	2277	2277
query89	472	415	384	384
query90	1995	158	154	154
query91	177	171	144	144
query92	69	69	65	65
query93	1098	901	560	560
query94	545	308	272	272
query95	577	331	356	331
query96	591	467	213	213
query97	2277	2301	2200	2200
query98	203	203	203	203
query99	1301	1296	1219	1219
Total cold run time: 252388 ms
Total hot run time: 175590 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.43 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 994180846dd28e9bdffe341fcf517654fa2351fd, data reload: false

query1	0.06	0.05	0.04
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.62	0.11	0.10
query5	0.27	0.26	0.26
query6	1.18	0.64	0.62
query7	0.04	0.03	0.03
query8	0.05	0.04	0.04
query9	0.59	0.51	0.50
query10	0.56	0.56	0.56
query11	0.15	0.11	0.12
query12	0.15	0.12	0.12
query13	0.63	0.59	0.59
query14	1.00	0.98	0.99
query15	0.83	0.81	0.79
query16	0.39	0.40	0.39
query17	1.03	1.05	1.03
query18	0.24	0.22	0.22
query19	1.85	1.89	1.80
query20	0.02	0.01	0.01
query21	15.75	0.30	0.15
query22	5.26	0.05	0.05
query23	16.02	0.30	0.11
query24	1.60	1.07	0.32
query25	0.09	0.08	0.07
query26	0.13	0.13	0.13
query27	0.06	0.07	0.05
query28	4.99	1.23	1.03
query29	12.58	4.04	3.19
query30	0.28	0.14	0.12
query31	2.83	0.64	0.39
query32	3.24	0.55	0.46
query33	2.96	3.12	3.14
query34	17.01	5.12	4.54
query35	4.57	4.53	4.61
query36	0.64	0.51	0.49
query37	0.10	0.07	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.14	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.03
Total cold run time: 99.59 s
Total hot run time: 27.43 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 94.22% (163/173) 🎉
Increment coverage report
Complete coverage report

@feiniaofeiafei feiniaofeiafei force-pushed the repeat_rewrite branch 2 times, most recently from fa63eb8 to 7e6da4f Compare December 17, 2025 11:38
@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 36182 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3446a7d4f011b796316c60f49e9cdecc2ab79431, data reload: false

------ Round 1 ----------------------------------
q1	17617	4258	4064	4064
q2	2031	350	234	234
q3	10190	1330	734	734
q4	10221	867	319	319
q5	7551	2160	1878	1878
q6	184	170	136	136
q7	1018	853	718	718
q8	9344	1453	1129	1129
q9	7065	5347	5361	5347
q10	6838	2405	1973	1973
q11	498	324	299	299
q12	667	754	572	572
q13	17771	3625	3060	3060
q14	291	291	269	269
q15	593	509	505	505
q16	702	684	619	619
q17	709	792	554	554
q18	7475	7529	7687	7529
q19	1146	1015	671	671
q20	419	416	285	285
q21	4510	4232	4271	4232
q22	1154	1103	1055	1055
Total cold run time: 107994 ms
Total hot run time: 36182 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4275	4217	4205	4205
q2	346	406	322	322
q3	2498	2939	2479	2479
q4	1416	1950	1389	1389
q5	4600	4466	4415	4415
q6	203	171	124	124
q7	2007	1979	1821	1821
q8	2609	2544	2519	2519
q9	7529	7146	6948	6948
q10	2926	3133	2657	2657
q11	577	508	478	478
q12	620	695	558	558
q13	3226	3685	3024	3024
q14	272	294	275	275
q15	531	497	482	482
q16	612	649	615	615
q17	1117	1339	1389	1339
q18	7323	7128	7221	7128
q19	843	812	858	812
q20	1945	1991	1839	1839
q21	4836	4292	4191	4191
q22	1112	1004	991	991
Total cold run time: 51423 ms
Total hot run time: 48611 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 178228 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3446a7d4f011b796316c60f49e9cdecc2ab79431, data reload: false

query5	5062	597	440	440
query6	321	235	211	211
query7	4217	477	273	273
query8	338	256	249	249
query9	8779	2552	2568	2552
query10	564	366	332	332
query11	15286	14736	14812	14736
query12	177	117	118	117
query13	1265	498	422	422
query14	6436	3050	2807	2807
query14_1	2754	2668	2664	2664
query15	208	197	183	183
query16	913	489	447	447
query17	1140	712	594	594
query18	2644	454	354	354
query19	232	232	211	211
query20	125	115	114	114
query21	221	139	117	117
query22	3849	3968	3967	3967
query23	16603	16220	15876	15876
query23_1	16157	16168	16026	16026
query24	7414	1627	1221	1221
query24_1	1225	1243	1261	1243
query25	561	478	422	422
query26	1250	268	158	158
query27	2760	465	309	309
query28	4463	2121	2124	2121
query29	787	562	448	448
query30	310	246	211	211
query31	827	727	645	645
query32	78	69	67	67
query33	533	326	319	319
query34	905	901	551	551
query35	786	816	739	739
query36	865	896	817	817
query37	136	84	78	78
query38	2875	2898	2821	2821
query39	759	732	706	706
query39_1	709	700	713	700
query40	222	137	123	123
query41	72	63	63	63
query42	106	107	107	107
query43	432	454	411	411
query44	1325	745	746	745
query45	200	192	176	176
query46	873	977	612	612
query47	1660	1680	1614	1614
query48	315	324	250	250
query49	632	446	353	353
query50	665	293	218	218
query51	3823	3937	3824	3824
query52	110	111	97	97
query53	329	351	292	292
query54	291	257	250	250
query55	85	79	81	79
query56	298	290	286	286
query57	1162	1147	1081	1081
query58	271	257	249	249
query59	2360	2443	2371	2371
query60	319	302	298	298
query61	162	157	154	154
query62	727	676	627	627
query63	325	307	304	304
query64	4909	1290	1000	1000
query65	3999	3930	3975	3930
query66	1394	440	349	349
query67	15191	15237	15097	15097
query68	2467	1114	771	771
query69	443	359	325	325
query70	1063	1002	983	983
query71	314	304	294	294
query72	5615	4923	4754	4754
query73	463	528	306	306
query74	8777	8822	8591	8591
query75	3104	3172	2797	2797
query76	2705	1123	726	726
query77	373	407	297	297
query78	9544	9465	8790	8790
query79	2964	873	612	612
query80	1718	680	559	559
query81	575	273	239	239
query82	950	135	105	105
query83	338	256	240	240
query84	263	126	100	100
query85	970	510	475	475
query86	485	276	277	276
query87	3106	3003	2892	2892
query88	3386	2280	2273	2273
query89	473	423	398	398
query90	2140	161	162	161
query91	172	177	143	143
query92	87	69	64	64
query93	2252	875	558	558
query94	541	298	272	272
query95	565	379	310	310
query96	600	476	209	209
query97	2304	2343	2272	2272
query98	214	196	199	196
query99	1298	1292	1218	1218
Total cold run time: 256704 ms
Total hot run time: 178228 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.12 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3446a7d4f011b796316c60f49e9cdecc2ab79431, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.25	0.09	0.09
query4	1.61	0.12	0.11
query5	0.28	0.26	0.24
query6	1.18	0.63	0.64
query7	0.03	0.03	0.03
query8	0.06	0.05	0.05
query9	0.57	0.51	0.50
query10	0.57	0.54	0.55
query11	0.16	0.11	0.12
query12	0.14	0.12	0.12
query13	0.64	0.61	0.59
query14	0.98	0.97	0.97
query15	0.82	0.80	0.81
query16	0.44	0.42	0.40
query17	1.04	0.99	0.99
query18	0.23	0.21	0.22
query19	1.88	1.90	1.86
query20	0.02	0.01	0.01
query21	15.46	0.28	0.15
query22	4.82	0.06	0.05
query23	16.03	0.30	0.10
query24	1.09	0.57	0.19
query25	0.11	0.10	0.05
query26	0.14	0.13	0.14
query27	0.08	0.05	0.05
query28	3.47	1.22	1.04
query29	12.56	3.89	3.13
query30	0.28	0.13	0.12
query31	2.80	0.60	0.41
query32	3.24	0.54	0.47
query33	3.02	3.02	3.02
query34	16.71	5.24	4.49
query35	4.61	4.58	4.54
query36	0.69	0.52	0.49
query37	0.11	0.07	0.06
query38	0.06	0.04	0.04
query39	0.05	0.03	0.03
query40	0.17	0.14	0.13
query41	0.09	0.02	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 96.73 s
Total hot run time: 27.12 s

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 97.22% (175/180) 🎉
Increment coverage report
Complete coverage report

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 94.51% (172/182) 🎉
Increment coverage report
Complete coverage report

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 35538 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f705059f53e94e36b782586e41982d2f0ed0cfc7, data reload: false

------ Round 1 ----------------------------------
q1	17656	4284	4147	4147
q2	2052	364	245	245
q3	10152	1433	771	771
q4	10237	910	321	321
q5	7523	2125	1984	1984
q6	185	174	136	136
q7	988	882	721	721
q8	9351	1479	1203	1203
q9	7056	5324	5369	5324
q10	6781	2413	1971	1971
q11	576	336	304	304
q12	666	724	594	594
q13	17758	3695	3108	3108
q14	303	290	280	280
q15	584	518	511	511
q16	696	681	628	628
q17	679	739	588	588
q18	7790	7125	7157	7125
q19	894	954	607	607
q20	414	361	247	247
q21	4224	3964	3782	3782
q22	1037	1005	941	941
Total cold run time: 107602 ms
Total hot run time: 35538 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4101	4051	4067	4051
q2	340	410	338	338
q3	2135	2720	2322	2322
q4	1358	1790	1351	1351
q5	4212	4650	4757	4650
q6	223	170	134	134
q7	2052	1988	1789	1789
q8	2685	2544	2578	2544
q9	7811	7323	7580	7323
q10	3077	3298	2779	2779
q11	611	542	528	528
q12	696	792	590	590
q13	3605	3936	3391	3391
q14	286	303	301	301
q15	573	530	499	499
q16	679	680	655	655
q17	1181	1443	1452	1443
q18	8050	7775	7568	7568
q19	887	882	900	882
q20	2083	2081	1950	1950
q21	4887	4608	4349	4349
q22	1101	1035	994	994
Total cold run time: 52633 ms
Total hot run time: 50431 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 180381 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f705059f53e94e36b782586e41982d2f0ed0cfc7, data reload: false

query5	5306	607	449	449
query6	334	225	220	220
query7	4225	470	280	280
query8	315	249	247	247
query9	8777	2530	2528	2528
query10	579	367	333	333
query11	15568	14901	14735	14735
query12	190	131	117	117
query13	1251	508	417	417
query14	6632	3092	2832	2832
query14_1	2698	2700	2656	2656
query15	218	197	178	178
query16	847	461	455	455
query17	1122	711	610	610
query18	2711	487	360	360
query19	237	241	216	216
query20	124	111	117	111
query21	228	144	119	119
query22	3930	3913	3936	3913
query23	16628	16237	16098	16098
query23_1	16119	16101	16130	16101
query24	7292	1627	1288	1288
query24_1	1237	1244	1269	1244
query25	593	502	481	481
query26	1253	272	166	166
query27	2739	473	312	312
query28	4455	2167	2139	2139
query29	818	602	448	448
query30	338	243	209	209
query31	832	701	658	658
query32	79	71	68	68
query33	540	332	280	280
query34	892	903	543	543
query35	768	788	705	705
query36	858	925	797	797
query37	129	95	81	81
query38	2946	3037	3012	3012
query39	760	749	723	723
query39_1	706	730	714	714
query40	229	156	121	121
query41	68	63	61	61
query42	109	108	106	106
query43	440	444	416	416
query44	1377	744	740	740
query45	194	187	180	180
query46	903	980	609	609
query47	1678	1666	1614	1614
query48	322	327	246	246
query49	629	459	357	357
query50	666	302	225	225
query51	3799	3853	3823	3823
query52	104	108	100	100
query53	334	354	302	302
query54	287	255	245	245
query55	80	77	72	72
query56	332	298	320	298
query57	1153	1142	1124	1124
query58	268	259	258	258
query59	2441	2504	2420	2420
query60	309	311	290	290
query61	165	157	155	155
query62	716	723	658	658
query63	340	306	301	301
query64	4895	1302	1008	1008
query65	4062	3925	3986	3925
query66	1376	428	320	320
query67	15318	15455	15313	15313
query68	2460	1103	775	775
query69	439	344	311	311
query70	1065	1004	916	916
query71	327	337	278	278
query72	5388	4888	4725	4725
query73	470	551	308	308
query74	8896	8854	8866	8854
query75	3099	3178	2797	2797
query76	2688	1150	763	763
query77	346	415	300	300
query78	9599	9643	8836	8836
query79	2123	918	611	611
query80	1454	645	568	568
query81	556	270	245	245
query82	1000	136	102	102
query83	349	265	236	236
query84	263	117	97	97
query85	1000	504	463	463
query86	409	300	304	300
query87	3113	3298	3104	3104
query88	3375	2302	2296	2296
query89	483	433	403	403
query90	1945	169	165	165
query91	173	165	144	144
query92	67	74	68	68
query93	1097	940	570	570
query94	548	294	294	294
query95	588	335	369	335
query96	600	498	220	220
query97	2276	2274	2233	2233
query98	212	201	201	201
query99	1368	1342	1358	1342
Total cold run time: 256239 ms
Total hot run time: 180381 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f705059f53e94e36b782586e41982d2f0ed0cfc7, data reload: false

query1	0.06	0.04	0.05
query2	0.10	0.04	0.05
query3	0.25	0.09	0.08
query4	1.61	0.12	0.12
query5	0.26	0.26	0.26
query6	1.17	0.64	0.64
query7	0.03	0.03	0.02
query8	0.06	0.05	0.05
query9	0.57	0.51	0.51
query10	0.56	0.54	0.54
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.62	0.61	0.61
query14	1.00	0.98	1.00
query15	0.84	0.81	0.81
query16	0.38	0.38	0.42
query17	1.07	1.05	1.00
query18	0.23	0.22	0.21
query19	1.93	1.84	1.72
query20	0.02	0.01	0.01
query21	15.44	0.30	0.14
query22	4.71	0.06	0.05
query23	15.88	0.28	0.12
query24	2.17	0.26	0.24
query25	0.10	0.06	0.06
query26	0.14	0.13	0.14
query27	0.05	0.05	0.06
query28	3.97	1.23	1.02
query29	12.66	4.03	3.22
query30	0.30	0.13	0.14
query31	2.81	0.65	0.41
query32	3.22	0.55	0.47
query33	3.10	3.01	3.04
query34	16.81	5.17	4.54
query35	4.64	4.58	4.57
query36	0.66	0.49	0.49
query37	0.12	0.07	0.07
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.16	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.03	0.03
Total cold run time: 98.28 s
Total hot run time: 27.25 s

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 94.54% (173/183) 🎉
Increment coverage report
Complete coverage report

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 36494 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 02bc423ecdc1dc28f7972259cddbb83ba2b0f6cd, data reload: false

------ Round 1 ----------------------------------
q1	17607	4221	4072	4072
q2	1989	351	239	239
q3	10227	1371	765	765
q4	10199	855	311	311
q5	7532	2185	1912	1912
q6	191	167	137	137
q7	1006	866	712	712
q8	9377	1476	1143	1143
q9	7280	5327	5361	5327
q10	6856	2403	2010	2010
q11	534	349	314	314
q12	702	722	586	586
q13	17778	3713	3056	3056
q14	289	301	272	272
q15	591	530	517	517
q16	710	681	638	638
q17	707	793	569	569
q18	7734	7732	7904	7732
q19	1547	1043	665	665
q20	433	393	251	251
q21	4688	4222	4378	4222
q22	1167	1064	1044	1044
Total cold run time: 109144 ms
Total hot run time: 36494 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4365	4287	4222	4222
q2	344	419	335	335
q3	2328	2858	2554	2554
q4	1455	1830	1426	1426
q5	4540	4621	4513	4513
q6	218	168	126	126
q7	2030	1986	1817	1817
q8	2758	2588	2589	2588
q9	7580	7562	7533	7533
q10	3062	3214	2664	2664
q11	599	518	504	504
q12	627	721	554	554
q13	3276	3664	3084	3084
q14	299	279	253	253
q15	529	487	484	484
q16	628	661	607	607
q17	1113	1311	1303	1303
q18	7231	7061	7184	7061
q19	839	796	853	796
q20	1883	1955	1839	1839
q21	4628	4268	4209	4209
q22	1103	1019	956	956
Total cold run time: 51435 ms
Total hot run time: 49428 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 179283 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 02bc423ecdc1dc28f7972259cddbb83ba2b0f6cd, data reload: false

query5	4397	610	488	488
query6	340	238	220	220
query7	4215	465	274	274
query8	303	253	263	253
query9	8752	2526	2571	2526
query10	495	404	325	325
query11	15599	15240	14715	14715
query12	183	118	117	117
query13	1275	511	387	387
query14	6166	3008	2785	2785
query14_1	2634	2648	2661	2648
query15	206	193	173	173
query16	822	449	430	430
query17	1134	698	588	588
query18	2486	424	336	336
query19	240	223	205	205
query20	119	112	113	112
query21	217	137	113	113
query22	4082	4028	3847	3847
query23	16546	16100	15914	15914
query23_1	16008	16092	16018	16018
query24	7396	1657	1237	1237
query24_1	1227	1223	1256	1223
query25	549	463	441	441
query26	1257	274	161	161
query27	2768	481	308	308
query28	4473	2122	2120	2120
query29	795	555	480	480
query30	322	246	209	209
query31	839	704	628	628
query32	83	75	67	67
query33	558	357	329	329
query34	917	899	545	545
query35	761	820	723	723
query36	877	908	824	824
query37	139	100	85	85
query38	3018	3032	2993	2993
query39	799	739	734	734
query39_1	719	705	706	705
query40	230	145	144	144
query41	73	68	67	67
query42	112	109	107	107
query43	431	423	416	416
query44	1351	749	744	744
query45	197	195	187	187
query46	874	989	610	610
query47	1703	1727	1690	1690
query48	329	330	258	258
query49	655	451	363	363
query50	671	310	227	227
query51	3776	3982	3787	3787
query52	111	111	100	100
query53	325	349	294	294
query54	309	279	270	270
query55	77	80	77	77
query56	310	324	317	317
query57	1173	1172	1088	1088
query58	284	261	255	255
query59	2333	2460	2388	2388
query60	334	327	320	320
query61	213	191	192	191
query62	750	736	672	672
query63	329	297	333	297
query64	4997	1310	1021	1021
query65	4035	3968	3961	3961
query66	1456	423	314	314
query67	15342	15454	15206	15206
query68	2612	1109	764	764
query69	435	336	301	301
query70	1075	1015	1018	1015
query71	318	303	275	275
query72	5536	4852	4881	4852
query73	469	539	308	308
query74	9036	8912	8573	8573
query75	3094	3152	2785	2785
query76	2748	1127	731	731
query77	372	387	296	296
query78	9553	9828	8817	8817
query79	1196	873	635	635
query80	1351	657	560	560
query81	569	266	234	234
query82	981	131	106	106
query83	350	257	244	244
query84	259	121	97	97
query85	974	507	465	465
query86	403	296	306	296
query87	3178	3117	3033	3033
query88	3405	2285	2295	2285
query89	473	420	385	385
query90	1974	167	156	156
query91	172	164	141	141
query92	77	65	62	62
query93	1166	934	563	563
query94	524	312	283	283
query95	576	338	368	338
query96	597	468	216	216
query97	2287	2264	2220	2220
query98	217	191	190	190
query99	1317	1328	1305	1305
Total cold run time: 253910 ms
Total hot run time: 179283 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.15 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 02bc423ecdc1dc28f7972259cddbb83ba2b0f6cd, data reload: false

query1	0.06	0.05	0.05
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.63	0.11	0.11
query5	0.27	0.27	0.26
query6	1.16	0.65	0.63
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.56	0.51	0.51
query10	0.55	0.54	0.53
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.62	0.61	0.60
query14	0.99	0.99	0.99
query15	0.81	0.80	0.80
query16	0.39	0.40	0.42
query17	1.00	1.04	1.01
query18	0.23	0.21	0.22
query19	1.92	1.71	1.86
query20	0.02	0.01	0.02
query21	15.45	0.28	0.13
query22	4.97	0.05	0.04
query23	16.05	0.29	0.10
query24	1.08	0.29	0.57
query25	0.11	0.08	0.06
query26	0.15	0.13	0.13
query27	0.07	0.05	0.06
query28	4.41	1.23	1.02
query29	12.58	3.95	3.16
query30	0.29	0.13	0.12
query31	2.82	0.63	0.40
query32	3.24	0.54	0.45
query33	2.95	3.05	3.05
query34	16.97	5.18	4.51
query35	4.56	4.54	4.61
query36	0.67	0.49	0.51
query37	0.10	0.06	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.87 s
Total hot run time: 27.15 s

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 92.35% (169/183) 🎉
Increment coverage report
Complete coverage report

@feiniaofeiafei
Copy link
Contributor Author

run cloud_p0

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 94.54% (173/183) 🎉
Increment coverage report
Complete coverage report

@morrySnow
Copy link
Contributor

add desc

@doris-robot
Copy link

TPC-DS: Total hot run time: 172810 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 19308d6848eb4383cee81dce6e6e29775cc38259, data reload: false

query5	4417	572	418	418
query6	325	219	214	214
query7	4224	453	248	248
query8	333	248	233	233
query9	8747	2606	2611	2606
query10	507	363	325	325
query11	15127	14957	14944	14944
query12	177	120	108	108
query13	1269	484	378	378
query14	5617	2981	2759	2759
query14_1	2607	2636	2616	2616
query15	209	198	182	182
query16	967	463	466	463
query17	905	691	587	587
query18	2427	459	330	330
query19	221	215	182	182
query20	129	119	110	110
query21	220	134	116	116
query22	3890	3922	4062	3922
query23	15945	15735	15335	15335
query23_1	15376	15444	15373	15373
query24	7345	1547	1197	1197
query24_1	1147	1167	1166	1166
query25	516	448	409	409
query26	1251	258	150	150
query27	2780	435	280	280
query28	4602	2105	2081	2081
query29	752	531	433	433
query30	318	237	204	204
query31	787	616	561	561
query32	73	66	61	61
query33	523	325	277	277
query34	890	875	520	520
query35	737	748	670	670
query36	853	880	810	810
query37	140	89	110	89
query38	2749	2714	2646	2646
query39	766	743	736	736
query39_1	699	722	694	694
query40	215	129	113	113
query41	70	61	60	60
query42	109	103	100	100
query43	494	470	431	431
query44	1298	706	710	706
query45	184	195	174	174
query46	829	935	593	593
query47	1473	1434	1405	1405
query48	307	321	230	230
query49	645	406	319	319
query50	630	258	211	211
query51	3804	3729	3674	3674
query52	103	103	95	95
query53	283	316	262	262
query54	282	260	240	240
query55	78	74	71	71
query56	290	292	283	283
query57	999	1003	904	904
query58	270	248	244	244
query59	2056	2077	2002	2002
query60	314	305	291	291
query61	179	161	148	148
query62	379	368	320	320
query63	310	271	266	266
query64	4936	1286	980	980
query65	3760	3702	3760	3702
query66	1454	411	289	289
query67	15437	15558	15418	15418
query68	2505	1069	731	731
query69	430	351	302	302
query70	977	989	962	962
query71	311	306	262	262
query72	5461	3480	3649	3480
query73	579	708	303	303
query74	8777	8756	8500	8500
query75	2778	2810	2496	2496
query76	2703	1045	642	642
query77	347	375	279	279
query78	9648	9893	9117	9117
query79	1059	913	591	591
query80	700	593	556	556
query81	482	256	240	240
query82	1376	143	106	106
query83	360	249	234	234
query84	301	116	93	93
query85	863	511	449	449
query86	365	321	310	310
query87	2836	2888	2793	2793
query88	3154	2191	2171	2171
query89	384	341	320	320
query90	1940	145	144	144
query91	176	161	140	140
query92	69	64	67	64
query93	920	872	527	527
query94	456	311	286	286
query95	578	379	310	310
query96	586	447	198	198
query97	2304	2374	2289	2289
query98	206	202	194	194
query99	610	580	561	561
Total cold run time: 245312 ms
Total hot run time: 172810 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 26.81 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 19308d6848eb4383cee81dce6e6e29775cc38259, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.25	0.08	0.08
query4	1.61	0.11	0.11
query5	0.27	0.25	0.26
query6	1.14	0.66	0.66
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.56	0.50	0.49
query10	0.54	0.55	0.54
query11	0.14	0.09	0.10
query12	0.15	0.11	0.11
query13	0.59	0.57	0.59
query14	0.96	0.95	0.92
query15	0.79	0.77	0.78
query16	0.40	0.39	0.40
query17	0.99	1.00	1.05
query18	0.23	0.22	0.21
query19	2.00	1.88	1.86
query20	0.02	0.03	0.01
query21	15.45	0.27	0.13
query22	5.29	0.05	0.05
query23	15.83	0.28	0.10
query24	1.12	0.66	0.31
query25	0.07	0.09	0.08
query26	0.14	0.13	0.13
query27	0.07	0.07	0.04
query28	3.26	1.08	0.88
query29	12.53	3.86	3.17
query30	0.29	0.13	0.11
query31	2.82	0.64	0.40
query32	3.24	0.56	0.44
query33	3.01	3.04	3.16
query34	16.75	5.13	4.43
query35	4.49	4.46	4.45
query36	0.66	0.50	0.48
query37	0.11	0.06	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.10	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 96.46 s
Total hot run time: 26.81 s

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 96.41% (215/223) 🎉
Increment coverage report
Complete coverage report

@feiniaofeiafei
Copy link
Contributor Author

run cloud_p0

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 96.41% (215/223) 🎉
Increment coverage report
Complete coverage report

@feiniaofeiafei
Copy link
Contributor Author

run cloud_p0

@feiniaofeiafei
Copy link
Contributor Author

run nonConcurrent

@feiniaofeiafei
Copy link
Contributor Author

run buildall

@feiniaofeiafei
Copy link
Contributor Author

run feut

@doris-robot
Copy link

TPC-H: Total hot run time: 31456 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fce2d9e91e224126f85d1b14ea3120609b6e4416, data reload: false

------ Round 1 ----------------------------------
q1	17636	4202	4039	4039
q2	2033	406	243	243
q3	10139	1327	716	716
q4	10199	873	315	315
q5	7493	2100	1846	1846
q6	195	171	140	140
q7	955	798	663	663
q8	9265	1386	1103	1103
q9	4969	4560	4625	4560
q10	6824	1815	1402	1402
q11	514	286	275	275
q12	718	718	587	587
q13	17782	3804	3105	3105
q14	286	291	268	268
q15	589	505	502	502
q16	700	677	648	648
q17	681	779	568	568
q18	7130	6414	6478	6414
q19	1097	966	593	593
q20	397	360	257	257
q21	2978	2417	2262	2262
q22	1048	992	950	950
Total cold run time: 103628 ms
Total hot run time: 31456 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4137	4019	4027	4019
q2	327	392	338	338
q3	2121	2562	2210	2210
q4	1328	1776	1380	1380
q5	4111	3990	4018	3990
q6	212	169	130	130
q7	1850	1814	1635	1635
q8	2834	2449	2395	2395
q9	7227	7144	7127	7127
q10	2514	2827	2322	2322
q11	566	481	462	462
q12	706	758	630	630
q13	3615	4015	3345	3345
q14	296	304	412	304
q15	558	505	503	503
q16	662	689	621	621
q17	1161	1348	1514	1348
q18	8023	7757	7514	7514
q19	851	841	867	841
q20	2004	2058	1934	1934
q21	4863	4415	4432	4415
q22	1099	1054	955	955
Total cold run time: 51065 ms
Total hot run time: 48418 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173377 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fce2d9e91e224126f85d1b14ea3120609b6e4416, data reload: false

query5	4434	589	445	445
query6	328	222	218	218
query7	4243	452	271	271
query8	319	246	226	226
query9	8725	2656	2653	2653
query10	508	360	307	307
query11	15118	15108	14811	14811
query12	195	118	124	118
query13	1247	494	356	356
query14	6043	2973	2763	2763
query14_1	2651	2645	2615	2615
query15	197	186	170	170
query16	1025	458	445	445
query17	1085	651	552	552
query18	2475	437	338	338
query19	223	226	202	202
query20	122	120	118	118
query21	219	138	127	127
query22	3802	4150	3905	3905
query23	15832	15651	15260	15260
query23_1	15335	15627	15450	15450
query24	7459	1527	1179	1179
query24_1	1192	1178	1188	1178
query25	533	435	399	399
query26	1247	268	150	150
query27	2780	449	282	282
query28	4555	2136	2132	2132
query29	795	563	450	450
query30	311	243	210	210
query31	803	625	570	570
query32	82	66	68	66
query33	530	334	285	285
query34	867	887	540	540
query35	710	755	667	667
query36	784	895	796	796
query37	125	92	78	78
query38	2685	2680	2708	2680
query39	763	766	731	731
query39_1	708	746	719	719
query40	214	136	116	116
query41	69	63	62	62
query42	104	100	100	100
query43	472	460	423	423
query44	1304	718	719	718
query45	183	186	183	183
query46	864	955	585	585
query47	1409	1488	1315	1315
query48	303	322	245	245
query49	607	412	334	334
query50	647	269	200	200
query51	3751	3762	3707	3707
query52	110	108	97	97
query53	303	324	272	272
query54	292	251	255	251
query55	75	75	77	75
query56	284	290	296	290
query57	1035	958	922	922
query58	265	258	244	244
query59	2058	2095	1988	1988
query60	335	325	315	315
query61	198	185	188	185
query62	385	368	373	368
query63	300	279	276	276
query64	5118	1448	1155	1155
query65	3801	3754	3770	3754
query66	1438	421	323	323
query67	15605	15575	15364	15364
query68	2452	1083	743	743
query69	429	364	319	319
query70	1050	968	949	949
query71	324	309	293	293
query72	5678	3439	3442	3439
query73	614	724	297	297
query74	8709	8746	8643	8643
query75	2726	2778	2440	2440
query76	2724	1068	653	653
query77	340	383	276	276
query78	9711	9923	9148	9148
query79	1086	945	576	576
query80	1273	610	470	470
query81	541	260	234	234
query82	1007	142	109	109
query83	326	251	244	244
query84	261	117	100	100
query85	910	507	454	454
query86	417	335	329	329
query87	2871	2902	2730	2730
query88	3192	2218	2192	2192
query89	379	364	342	342
query90	1976	154	168	154
query91	170	164	139	139
query92	71	68	61	61
query93	957	872	518	518
query94	651	313	289	289
query95	578	327	360	327
query96	579	462	204	204
query97	2342	2364	2360	2360
query98	224	201	198	198
query99	571	623	502	502
Total cold run time: 246741 ms
Total hot run time: 173377 ms

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 97.31% (217/223) 🎉
Increment coverage report
Complete coverage report

@feiniaofeiafei
Copy link
Contributor Author

run feut

@feiniaofeiafei
Copy link
Contributor Author

run p0

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage 97.31% (217/223) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants