Skip to content

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Jan 1, 2026

What problem does this PR solve?

cost based rewrite job may contaminate StatementContext.rewrittenCteProducer

fix solution: clone current rewrittenCteProducer, and restore it after each call to getCost(...)

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
Copy link
Contributor Author

englefly commented Jan 1, 2026

run buildall

@englefly englefly force-pushed the cte-producer-costrewrite branch from 6e97d27 to 073c151 Compare January 1, 2026 03:21
@englefly
Copy link
Contributor Author

englefly commented Jan 1, 2026

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32774 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 073c15115f82ffbe0fe51009f6cfcfc6e8eaa237, data reload: false

------ Round 1 ----------------------------------
q1	17681	4284	4053	4053
q2	2052	364	244	244
q3	10168	1280	754	754
q4	10218	902	345	345
q5	7546	2179	1958	1958
q6	189	176	136	136
q7	965	795	673	673
q8	9298	1424	1221	1221
q9	5195	4740	4806	4740
q10	6797	1801	1412	1412
q11	487	318	296	296
q12	696	726	582	582
q13	17795	3804	3162	3162
q14	288	291	273	273
q15	576	510	510	510
q16	713	684	651	651
q17	699	754	652	652
q18	6769	6370	7183	6370
q19	1153	1026	641	641
q20	423	390	274	274
q21	3390	2755	2881	2755
q22	1152	1102	1072	1072
Total cold run time: 104250 ms
Total hot run time: 32774 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4454	4257	4232	4232
q2	328	422	343	343
q3	2386	2746	2334	2334
q4	1417	1863	1461	1461
q5	4560	4257	4291	4257
q6	219	164	127	127
q7	1978	2002	1771	1771
q8	2528	2352	2405	2352
q9	7305	7111	7170	7111
q10	2466	2788	2277	2277
q11	549	506	458	458
q12	768	763	627	627
q13	3644	3985	3440	3440
q14	293	311	276	276
q15	542	497	488	488
q16	630	682	636	636
q17	1066	1278	1265	1265
q18	7486	7408	7060	7060
q19	829	824	852	824
q20	1889	1957	1834	1834
q21	4575	4273	4183	4183
q22	1095	1020	996	996
Total cold run time: 51007 ms
Total hot run time: 48352 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173200 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 073c15115f82ffbe0fe51009f6cfcfc6e8eaa237, data reload: false

query5	4556	607	459	459
query6	338	231	211	211
query7	4215	452	281	281
query8	327	247	255	247
query9	8822	2651	2645	2645
query10	511	357	324	324
query11	15456	15195	14981	14981
query12	178	122	113	113
query13	1263	504	381	381
query14	6247	3028	2776	2776
query14_1	2708	2651	2740	2651
query15	207	201	178	178
query16	1010	474	510	474
query17	1068	673	559	559
query18	2602	440	344	344
query19	228	216	191	191
query20	121	120	113	113
query21	214	142	121	121
query22	3790	4099	3902	3902
query23	16068	15707	15366	15366
query23_1	15595	15367	15454	15367
query24	7443	1580	1211	1211
query24_1	1235	1203	1210	1203
query25	539	439	422	422
query26	1244	269	161	161
query27	2764	474	297	297
query28	4512	2205	2205	2205
query29	804	533	441	441
query30	310	237	219	219
query31	818	631	554	554
query32	75	77	65	65
query33	536	331	283	283
query34	926	879	542	542
query35	771	791	707	707
query36	867	888	800	800
query37	137	95	79	79
query38	2765	2706	2647	2647
query39	779	757	715	715
query39_1	693	703	724	703
query40	220	139	119	119
query41	68	62	62	62
query42	106	106	110	106
query43	442	486	425	425
query44	1400	767	756	756
query45	187	185	175	175
query46	901	993	621	621
query47	1450	1456	1432	1432
query48	318	335	262	262
query49	614	410	328	328
query50	657	278	213	213
query51	3754	3786	3732	3732
query52	104	107	94	94
query53	302	327	275	275
query54	283	270	247	247
query55	80	84	75	75
query56	294	290	295	290
query57	1006	1019	907	907
query58	270	252	254	252
query59	2023	2261	2019	2019
query60	336	328	305	305
query61	196	191	191	191
query62	399	363	343	343
query63	304	272	278	272
query64	5168	1432	1111	1111
query65	3826	3716	3749	3716
query66	1428	448	327	327
query67	15953	14851	14491	14491
query68	4982	1057	751	751
query69	505	353	315	315
query70	1055	884	940	884
query71	362	323	283	283
query72	6231	3507	3521	3507
query73	790	760	323	323
query74	8813	8718	8564	8564
query75	2864	2857	2469	2469
query76	3873	1111	689	689
query77	521	377	282	282
query78	9722	9853	9198	9198
query79	1683	884	622	622
query80	688	582	489	489
query81	504	263	227	227
query82	218	141	109	109
query83	267	257	242	242
query84	263	123	105	105
query85	903	527	466	466
query86	393	319	318	318
query87	2825	2885	2732	2732
query88	3821	2294	2290	2290
query89	405	348	332	332
query90	2225	164	157	157
query91	171	168	142	142
query92	86	70	65	65
query93	1992	944	562	562
query94	573	337	298	298
query95	578	385	312	312
query96	589	484	221	221
query97	2297	2405	2301	2301
query98	247	202	200	200
query99	585	572	520	520
Total cold run time: 255395 ms
Total hot run time: 173200 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 26.79 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 073c15115f82ffbe0fe51009f6cfcfc6e8eaa237, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.05	0.05
query3	0.26	0.08	0.09
query4	1.61	0.12	0.11
query5	0.27	0.25	0.26
query6	1.14	0.66	0.64
query7	0.04	0.02	0.02
query8	0.05	0.04	0.04
query9	0.58	0.50	0.49
query10	0.56	0.56	0.55
query11	0.16	0.11	0.12
query12	0.16	0.13	0.14
query13	0.61	0.61	0.60
query14	0.99	1.00	0.96
query15	0.80	0.78	0.82
query16	0.42	0.42	0.41
query17	1.06	1.04	1.04
query18	0.22	0.21	0.22
query19	1.86	1.78	1.84
query20	0.02	0.02	0.01
query21	15.47	0.28	0.14
query22	4.86	0.05	0.05
query23	16.25	0.29	0.10
query24	1.34	0.75	0.19
query25	0.12	0.08	0.07
query26	0.14	0.13	0.14
query27	0.10	0.08	0.06
query28	5.18	1.07	0.88
query29	12.59	3.92	3.17
query30	0.30	0.17	0.11
query31	2.82	0.61	0.38
query32	3.23	0.55	0.46
query33	3.08	2.98	2.99
query34	16.65	5.07	4.48
query35	4.48	4.54	4.46
query36	0.66	0.51	0.49
query37	0.10	0.06	0.07
query38	0.07	0.04	0.04
query39	0.05	0.03	0.03
query40	0.17	0.15	0.13
query41	0.09	0.04	0.03
query42	0.04	0.03	0.02
query43	0.05	0.04	0.03
Total cold run time: 98.8 s
Total hot run time: 26.79 s

@englefly
Copy link
Contributor Author

englefly commented Jan 1, 2026

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32270 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ef065c65149203a5761c3548d72d363b6ab904c5, data reload: false

------ Round 1 ----------------------------------
q1	17593	4264	4193	4193
q2	2020	366	234	234
q3	10187	1271	733	733
q4	10215	842	338	338
q5	7550	2171	1907	1907
q6	196	178	146	146
q7	978	821	664	664
q8	9320	1442	1184	1184
q9	5225	4666	4678	4666
q10	6821	1796	1407	1407
q11	506	310	309	309
q12	679	718	612	612
q13	17776	3789	3090	3090
q14	300	295	292	292
q15	596	518	500	500
q16	701	675	646	646
q17	710	727	598	598
q18	6693	6659	6487	6487
q19	1100	968	619	619
q20	440	378	252	252
q21	3139	2541	2414	2414
q22	1068	998	979	979
Total cold run time: 103813 ms
Total hot run time: 32270 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4098	4031	4039	4031
q2	321	403	338	338
q3	2087	2619	2238	2238
q4	1336	1760	1331	1331
q5	4079	3954	4093	3954
q6	207	173	132	132
q7	1823	1774	2028	1774
q8	2573	2454	2399	2399
q9	7193	7163	7133	7133
q10	2607	2737	2330	2330
q11	582	471	471	471
q12	703	802	651	651
q13	3698	3984	3477	3477
q14	304	298	270	270
q15	542	505	509	505
q16	657	687	671	671
q17	1167	1313	1584	1313
q18	8081	8021	7541	7541
q19	941	912	949	912
q20	2082	2073	1941	1941
q21	4878	4547	4166	4166
q22	1094	1029	972	972
Total cold run time: 51053 ms
Total hot run time: 48550 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 172742 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ef065c65149203a5761c3548d72d363b6ab904c5, data reload: false

query5	4817	584	464	464
query6	335	223	216	216
query7	4221	464	256	256
query8	348	247	241	241
query9	8773	2632	2617	2617
query10	526	385	303	303
query11	15189	15237	14852	14852
query12	171	114	110	110
query13	1248	499	385	385
query14	6257	2960	2783	2783
query14_1	2719	2649	2711	2649
query15	199	196	180	180
query16	981	476	449	449
query17	1098	660	557	557
query18	2494	418	328	328
query19	215	213	184	184
query20	123	113	111	111
query21	213	133	120	120
query22	3858	3956	3782	3782
query23	15727	15485	15235	15235
query23_1	15459	15392	15206	15206
query24	7447	1572	1208	1208
query24_1	1194	1214	1227	1214
query25	525	436	395	395
query26	1238	260	155	155
query27	2763	464	290	290
query28	4593	2168	2169	2168
query29	790	541	444	444
query30	307	240	205	205
query31	796	631	552	552
query32	78	69	67	67
query33	527	328	281	281
query34	923	867	547	547
query35	763	785	716	716
query36	839	865	768	768
query37	123	94	87	87
query38	2770	2670	2626	2626
query39	767	738	750	738
query39_1	728	704	719	704
query40	217	133	116	116
query41	66	61	62	61
query42	102	107	103	103
query43	446	427	425	425
query44	1322	743	758	743
query45	188	188	179	179
query46	882	964	615	615
query47	1447	1445	1305	1305
query48	339	332	252	252
query49	610	408	328	328
query50	636	279	204	204
query51	3757	3773	3708	3708
query52	102	107	97	97
query53	292	327	275	275
query54	278	249	247	247
query55	75	77	72	72
query56	290	300	283	283
query57	1042	1029	932	932
query58	261	245	250	245
query59	2139	2204	1956	1956
query60	342	311	292	292
query61	161	158	170	158
query62	391	346	329	329
query63	302	266	275	266
query64	5052	1408	1143	1143
query65	3798	3745	3605	3605
query66	1440	446	316	316
query67	15109	15147	14983	14983
query68	3606	1044	734	734
query69	493	344	301	301
query70	1063	942	948	942
query71	344	292	276	276
query72	6098	3493	3424	3424
query73	775	709	313	313
query74	8745	8709	8544	8544
query75	2838	2854	2494	2494
query76	3873	1040	652	652
query77	522	355	273	273
query78	9663	9768	9135	9135
query79	960	918	605	605
query80	1144	590	495	495
query81	543	267	241	241
query82	408	142	107	107
query83	388	254	246	246
query84	256	115	105	105
query85	920	538	451	451
query86	390	316	301	301
query87	2796	2859	2853	2853
query88	3213	2284	2272	2272
query89	380	362	322	322
query90	1990	153	140	140
query91	171	166	148	148
query92	69	66	62	62
query93	1019	948	563	563
query94	637	323	278	278
query95	573	323	351	323
query96	586	453	206	206
query97	2298	2372	2315	2315
query98	209	215	211	211
query99	588	614	505	505
Total cold run time: 250043 ms
Total hot run time: 172742 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ef065c65149203a5761c3548d72d363b6ab904c5, data reload: false

query1	0.05	0.04	0.05
query2	0.09	0.05	0.05
query3	0.26	0.08	0.08
query4	1.60	0.11	0.11
query5	0.27	0.25	0.26
query6	1.15	0.66	0.66
query7	0.03	0.02	0.03
query8	0.06	0.04	0.05
query9	0.58	0.50	0.51
query10	0.56	0.53	0.55
query11	0.16	0.11	0.11
query12	0.17	0.13	0.13
query13	0.62	0.61	0.61
query14	0.99	0.98	0.99
query15	0.80	0.81	0.78
query16	0.38	0.39	0.40
query17	1.04	1.05	1.01
query18	0.23	0.22	0.21
query19	1.91	1.80	1.75
query20	0.02	0.01	0.02
query21	16.41	0.30	0.13
query22	4.66	0.05	0.05
query23	16.05	0.28	0.10
query24	0.97	0.57	0.63
query25	0.12	0.08	0.06
query26	0.14	0.14	0.14
query27	0.09	0.06	0.05
query28	4.20	1.04	0.88
query29	12.60	3.94	3.14
query30	0.28	0.13	0.14
query31	2.81	0.59	0.39
query32	3.23	0.56	0.46
query33	3.05	2.96	3.04
query34	16.49	5.07	4.44
query35	4.49	4.46	4.52
query36	0.67	0.51	0.49
query37	0.11	0.07	0.07
query38	0.07	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.14	0.14
query41	0.08	0.03	0.03
query42	0.05	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.78 s
Total hot run time: 27.06 s

@englefly englefly force-pushed the cte-producer-costrewrite branch from ef065c6 to 4438cb1 Compare January 1, 2026 15:58
@englefly
Copy link
Contributor Author

englefly commented Jan 1, 2026

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32582 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4438cb164163c15256d4a9a55ecec71ceb694cf8, data reload: false

------ Round 1 ----------------------------------
q1	17633	4243	4039	4039
q2	2013	372	244	244
q3	10156	1274	759	759
q4	10229	845	320	320
q5	8107	2100	1963	1963
q6	219	171	139	139
q7	967	814	667	667
q8	9305	1458	1137	1137
q9	5291	4747	4680	4680
q10	6846	1816	1436	1436
q11	510	315	303	303
q12	728	739	583	583
q13	17790	3877	3090	3090
q14	294	294	273	273
q15	587	515	500	500
q16	697	682	627	627
q17	714	738	669	669
q18	6778	6488	7094	6488
q19	1662	1168	649	649
q20	418	372	273	273
q21	3389	2694	2782	2694
q22	1180	1103	1049	1049
Total cold run time: 105513 ms
Total hot run time: 32582 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4396	4267	4300	4267
q2	340	429	329	329
q3	2346	2834	2513	2513
q4	1480	1823	1620	1620
q5	4605	4220	4299	4220
q6	219	176	132	132
q7	1920	1955	1758	1758
q8	2546	2344	2451	2344
q9	7185	7131	7107	7107
q10	2447	2697	2592	2592
q11	560	471	445	445
q12	705	760	598	598
q13	3586	3819	3074	3074
q14	260	281	253	253
q15	520	491	474	474
q16	607	661	589	589
q17	1094	1243	1335	1243
q18	7401	7384	7294	7294
q19	878	849	862	849
q20	1950	1978	1805	1805
q21	4534	4302	4396	4302
q22	1136	1035	1006	1006
Total cold run time: 50715 ms
Total hot run time: 48814 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173742 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4438cb164163c15256d4a9a55ecec71ceb694cf8, data reload: false

query5	4871	580	435	435
query6	329	235	219	219
query7	4206	459	271	271
query8	331	249	256	249
query9	8783	2617	2620	2617
query10	528	376	308	308
query11	15165	15125	15006	15006
query12	184	113	114	113
query13	1256	502	391	391
query14	6366	3017	2732	2732
query14_1	2653	2631	2668	2631
query15	205	187	171	171
query16	1022	472	367	367
query17	1066	695	564	564
query18	2615	425	329	329
query19	222	219	189	189
query20	125	117	123	117
query21	215	137	114	114
query22	3879	3886	3814	3814
query23	16198	15634	15378	15378
query23_1	15498	15510	15375	15375
query24	7341	1594	1205	1205
query24_1	1198	1201	1228	1201
query25	518	428	422	422
query26	1242	272	155	155
query27	2766	451	334	334
query28	4502	2197	2199	2197
query29	748	528	424	424
query30	312	242	204	204
query31	785	621	552	552
query32	75	71	67	67
query33	514	338	289	289
query34	891	882	529	529
query35	773	778	700	700
query36	883	882	807	807
query37	132	91	75	75
query38	2762	2717	2662	2662
query39	792	737	739	737
query39_1	714	704	708	704
query40	220	132	113	113
query41	66	64	63	63
query42	110	104	101	101
query43	418	447	430	430
query44	1326	755	743	743
query45	186	182	176	176
query46	873	966	613	613
query47	1389	1457	1320	1320
query48	316	329	241	241
query49	613	418	321	321
query50	650	290	207	207
query51	3715	3792	3797	3792
query52	111	110	93	93
query53	298	326	275	275
query54	289	257	260	257
query55	78	76	75	75
query56	288	291	294	291
query57	1001	969	909	909
query58	266	248	248	248
query59	2060	2141	2095	2095
query60	320	313	309	309
query61	182	189	193	189
query62	405	368	309	309
query63	304	271	274	271
query64	5016	1422	1089	1089
query65	3747	3755	3707	3707
query66	1425	446	330	330
query67	15201	15822	15508	15508
query68	4729	1035	763	763
query69	503	362	320	320
query70	1029	893	979	893
query71	363	313	277	277
query72	6276	3354	3418	3354
query73	779	715	328	328
query74	8747	8805	8609	8609
query75	2837	2832	2464	2464
query76	3950	1062	645	645
query77	516	359	281	281
query78	9836	9986	9237	9237
query79	1145	912	623	623
query80	843	578	480	480
query81	490	270	227	227
query82	411	146	113	113
query83	269	258	236	236
query84	254	123	105	105
query85	893	512	455	455
query86	357	308	305	305
query87	2917	2842	2789	2789
query88	3196	2292	2263	2263
query89	380	357	315	315
query90	1942	155	144	144
query91	172	169	145	145
query92	70	69	63	63
query93	1102	901	565	565
query94	642	332	258	258
query95	578	382	302	302
query96	586	452	208	208
query97	2390	2398	2320	2320
query98	230	195	204	195
query99	583	572	548	548
Total cold run time: 251760 ms
Total hot run time: 173742 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 27.28 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4438cb164163c15256d4a9a55ecec71ceb694cf8, data reload: false

query1	0.05	0.04	0.04
query2	0.10	0.05	0.05
query3	0.26	0.08	0.08
query4	1.60	0.12	0.11
query5	0.28	0.27	0.25
query6	1.14	0.66	0.65
query7	0.04	0.02	0.03
query8	0.05	0.05	0.04
query9	0.57	0.50	0.49
query10	0.56	0.54	0.55
query11	0.15	0.10	0.11
query12	0.16	0.13	0.13
query13	0.62	0.59	0.59
query14	1.00	0.99	0.98
query15	0.81	0.82	0.79
query16	0.40	0.41	0.42
query17	1.06	0.99	1.00
query18	0.24	0.21	0.22
query19	1.82	1.82	1.83
query20	0.02	0.02	0.01
query21	15.46	0.27	0.15
query22	4.71	0.05	0.05
query23	15.82	0.28	0.11
query24	0.94	0.68	0.79
query25	0.12	0.08	0.06
query26	0.14	0.13	0.14
query27	0.06	0.06	0.06
query28	5.46	1.05	0.88
query29	12.57	3.91	3.12
query30	0.27	0.14	0.11
query31	2.81	0.63	0.37
query32	3.23	0.55	0.49
query33	3.08	3.03	3.05
query34	16.74	5.09	4.47
query35	4.46	4.43	4.55
query36	0.68	0.50	0.49
query37	0.10	0.06	0.06
query38	0.06	0.04	0.04
query39	0.05	0.03	0.03
query40	0.18	0.15	0.13
query41	0.09	0.04	0.03
query42	0.04	0.03	0.03
query43	0.04	0.04	0.03
Total cold run time: 98.04 s
Total hot run time: 27.28 s

@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (9/9) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants