Skip to content

[opt](memo) reduce memo size #59600

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
morrySnow wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

[opt](memo) reduce memo size #59600

morrySnow wants to merge 1 commit into from

Conversation

@morrySnow
Copy link
Contributor

@morrySnow morrySnow commented Jan 6, 2026

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Jan 6, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow morrySnow force-pushed the try_opt_memo branch 6 times, most recently from a020aba to 54d7ff2 Compare January 7, 2026 07:55
@morrySnow
Copy link
Contributor Author

morrySnow commented Jan 7, 2026

run buildall

@morrySnow
Copy link
Contributor Author

morrySnow commented Jan 7, 2026

run buildall

@morrySnow
Copy link
Contributor Author

morrySnow commented Jan 7, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 7, 2026

TPC-H: Total hot run time: 31807 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18a288a56cbbd910484e4888104c88c70be140e4, data reload: false

------ Round 1 ----------------------------------
q1	17656	4297	4055	4055
q2	2120	375	262	262
q3	10049	1246	744	744
q4	10214	905	327	327
q5	7541	2117	1831	1831
q6	207	177	145	145
q7	939	808	670	670
q8	9284	1449	1191	1191
q9	4863	4666	4511	4511
q10	6788	1788	1414	1414
q11	547	322	304	304
q12	699	734	585	585
q13	17773	3793	3076	3076
q14	304	294	270	270
q15	591	512	516	512
q16	679	667	632	632
q17	690	807	501	501
q18	6615	6365	6728	6365
q19	1291	1014	625	625
q20	490	463	270	270
q21	3273	2641	2511	2511
q22	1143	1066	1006	1006
Total cold run time: 103756 ms
Total hot run time: 31807 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4418	4293	4221	4221
q2	349	419	349	349
q3	2303	2801	2386	2386
q4	1429	1879	1406	1406
q5	4574	4300	4350	4300
q6	216	168	127	127
q7	1917	1997	1817	1817
q8	2573	2388	2380	2380
q9	6944	7026	7077	7026
q10	2482	2683	2383	2383
q11	536	480	465	465
q12	706	728	575	575
q13	3378	3764	3058	3058
q14	273	300	281	281
q15	532	500	500	500
q16	614	652	610	610
q17	1102	1350	1322	1322
q18	7392	7225	7167	7167
q19	835	774	824	774
q20	1893	1948	1873	1873
q21	4405	4251	4213	4213
q22	1129	1045	1012	1012
Total cold run time: 50000 ms
Total hot run time: 48245 ms

@doris-robot
Copy link

doris-robot commented Jan 7, 2026

TPC-DS: Total hot run time: 172228 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18a288a56cbbd910484e4888104c88c70be140e4, data reload: false

query5	4423	594	448	448
query6	346	257	229	229
query7	4222	466	277	277
query8	350	250	240	240
query9	8763	2673	2653	2653
query10	516	359	307	307
query11	15050	15089	14907	14907
query12	178	117	111	111
query13	1253	492	374	374
query14	6405	2954	2724	2724
query14_1	2631	2600	2638	2600
query15	204	192	172	172
query16	975	470	456	456
query17	1093	665	553	553
query18	2590	424	334	334
query19	222	225	205	205
query20	125	121	116	116
query21	211	139	124	124
query22	4117	4284	3948	3948
query23	16115	15730	15382	15382
query23_1	15459	15576	15546	15546
query24	7331	1563	1206	1206
query24_1	1198	1188	1184	1184
query25	549	461	405	405
query26	1244	268	159	159
query27	2770	454	289	289
query28	4493	2139	2141	2139
query29	748	550	506	506
query30	308	239	216	216
query31	812	641	553	553
query32	75	66	63	63
query33	525	326	269	269
query34	882	871	529	529
query35	728	772	686	686
query36	836	898	816	816
query37	138	88	77	77
query38	2701	2763	2570	2570
query39	782	761	737	737
query39_1	734	707	701	701
query40	217	129	117	117
query41	68	64	63	63
query42	102	98	99	98
query43	444	464	413	413
query44	1274	725	721	721
query45	188	182	173	173
query46	845	965	597	597
query47	1482	1488	1331	1331
query48	307	305	233	233
query49	599	445	325	325
query50	633	278	200	200
query51	3815	3820	3725	3725
query52	101	106	97	97
query53	294	319	267	267
query54	276	245	240	240
query55	76	73	69	69
query56	275	283	296	283
query57	987	1027	904	904
query58	262	248	250	248
query59	2043	2098	2102	2098
query60	317	307	289	289
query61	157	148	143	143
query62	414	357	308	308
query63	295	266	269	266
query64	4721	1248	944	944
query65	3773	3723	3695	3695
query66	1410	423	296	296
query67	15010	14711	15047	14711
query68	7636	967	702	702
query69	501	343	327	327
query70	1093	948	919	919
query71	369	296	277	277
query72	5987	3236	3302	3236
query73	760	724	302	302
query74	8773	8806	8633	8633
query75	2826	2783	2423	2423
query76	3937	1048	645	645
query77	524	382	270	270
query78	9679	9731	9101	9101
query79	1600	856	567	567
query80	592	568	448	448
query81	514	266	231	231
query82	501	141	107	107
query83	258	259	237	237
query84	251	114	102	102
query85	928	475	425	425
query86	395	284	323	284
query87	2839	2821	2724	2724
query88	4353	2217	2222	2217
query89	384	345	336	336
query90	2145	150	147	147
query91	163	163	138	138
query92	84	72	63	63
query93	1666	891	537	537
query94	567	313	271	271
query95	564	370	298	298
query96	580	475	210	210
query97	2344	2389	2286	2286
query98	209	202	202	202
query99	610	596	534	534
Total cold run time: 256405 ms
Total hot run time: 172228 ms

@doris-robot
Copy link

doris-robot commented Jan 7, 2026

ClickBench: Total hot run time: 26.79 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18a288a56cbbd910484e4888104c88c70be140e4, data reload: false

query1	0.05	0.05	0.05
query2	0.11	0.05	0.05
query3	0.26	0.08	0.08
query4	1.61	0.11	0.10
query5	0.26	0.25	0.25
query6	1.15	0.66	0.66
query7	0.04	0.02	0.02
query8	0.05	0.04	0.04
query9	0.58	0.49	0.50
query10	0.55	0.55	0.54
query11	0.15	0.10	0.11
query12	0.15	0.11	0.11
query13	0.60	0.58	0.58
query14	0.94	0.95	0.96
query15	0.80	0.79	0.77
query16	0.42	0.42	0.41
query17	0.98	1.07	1.07
query18	0.23	0.21	0.20
query19	1.94	1.81	1.86
query20	0.01	0.01	0.02
query21	15.45	0.28	0.14
query22	5.45	0.05	0.04
query23	15.69	0.29	0.10
query24	1.04	0.73	0.17
query25	0.08	0.06	0.06
query26	0.14	0.13	0.13
query27	0.06	0.08	0.06
query28	4.34	1.05	0.88
query29	12.54	3.92	3.14
query30	0.28	0.14	0.12
query31	2.82	0.64	0.39
query32	3.25	0.58	0.45
query33	2.99	3.08	3.10
query34	16.56	5.19	4.46
query35	4.52	4.44	4.51
query36	0.66	0.50	0.49
query37	0.11	0.07	0.06
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.17	0.15	0.14
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.04	0.04	0.03
Total cold run time: 97.3 s
Total hot run time: 26.79 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 7, 2026

FE UT Coverage Report

Increment line coverage 89.66% (52/58) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 7, 2026

FE Regression Coverage Report

Increment line coverage 89.66% (52/58) 🎉
Increment coverage report
Complete coverage report

@morrySnow
Copy link
Contributor Author

morrySnow commented Jan 8, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 8, 2026

TPC-H: Total hot run time: 31399 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18a288a56cbbd910484e4888104c88c70be140e4, data reload: false

------ Round 1 ----------------------------------
q1	17712	4163	4045	4045
q2	2183	379	237	237
q3	10263	1273	708	708
q4	10269	888	326	326
q5	7529	2004	1879	1879
q6	187	177	142	142
q7	892	788	657	657
q8	9719	1325	1154	1154
q9	5286	4673	4572	4572
q10	7054	1859	1471	1471
q11	655	306	284	284
q12	721	742	632	632
q13	17964	3776	3048	3048
q14	287	288	274	274
q15	611	512	500	500
q16	686	658	638	638
q17	735	785	528	528
q18	7177	6267	6344	6267
q19	1094	972	600	600
q20	408	357	254	254
q21	2984	2396	2244	2244
q22	1036	1026	939	939
Total cold run time: 105452 ms
Total hot run time: 31399 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4102	4050	4031	4031
q2	327	405	310	310
q3	2090	2570	2239	2239
q4	1334	1715	1341	1341
q5	4094	4012	4185	4012
q6	207	174	132	132
q7	1942	2043	1801	1801
q8	2544	2408	2404	2404
q9	7212	7156	7068	7068
q10	2552	2667	2278	2278
q11	558	484	463	463
q12	702	780	645	645
q13	3541	4126	3307	3307
q14	279	309	354	309
q15	542	509	504	504
q16	637	656	607	607
q17	1127	1265	1369	1265
q18	7984	7747	7992	7747
q19	867	816	818	816
q20	2069	2028	1909	1909
q21	4630	4696	4298	4298
q22	1108	1133	1064	1064
Total cold run time: 50448 ms
Total hot run time: 48550 ms

@doris-robot
Copy link

doris-robot commented Jan 8, 2026

TPC-DS: Total hot run time: 173782 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18a288a56cbbd910484e4888104c88c70be140e4, data reload: false

query5	4435	579	433	433
query6	326	233	243	233
query7	4215	454	266	266
query8	330	262	250	250
query9	8758	2621	2636	2621
query10	494	370	311	311
query11	15175	16064	15989	15989
query12	186	122	113	113
query13	1286	526	400	400
query14	6690	3083	2915	2915
query14_1	2757	2797	2738	2738
query15	227	208	188	188
query16	3981	521	482	482
query17	1182	727	604	604
query18	2246	429	346	346
query19	225	224	186	186
query20	136	116	114	114
query21	539	129	122	122
query22	3942	4105	3940	3940
query23	16065	15490	15274	15274
query23_1	15437	15411	15378	15378
query24	7547	1506	1161	1161
query24_1	1170	1169	1155	1155
query25	613	451	399	399
query26	1144	260	152	152
query27	2661	437	279	279
query28	4458	2094	2113	2094
query29	697	512	411	411
query30	341	228	198	198
query31	771	635	560	560
query32	71	64	60	60
query33	523	325	264	264
query34	881	866	506	506
query35	724	746	672	672
query36	879	870	848	848
query37	143	96	74	74
query38	2669	2690	2620	2620
query39	779	758	737	737
query39_1	743	711	704	704
query40	218	123	112	112
query41	63	57	58	57
query42	103	98	106	98
query43	422	467	410	410
query44	1290	744	709	709
query45	186	180	171	171
query46	833	973	584	584
query47	1337	1461	1338	1338
query48	311	338	236	236
query49	582	400	317	317
query50	628	265	203	203
query51	3742	3821	3737	3737
query52	102	108	98	98
query53	291	322	266	266
query54	278	248	240	240
query55	112	74	73	73
query56	280	294	273	273
query57	1011	991	928	928
query58	260	250	245	245
query59	2137	2080	2111	2080
query60	327	307	288	288
query61	151	141	149	141
query62	402	344	302	302
query63	301	266	271	266
query64	4330	1254	960	960
query65	3781	3685	3759	3685
query66	1272	390	288	288
query67	15326	15762	15175	15175
query68	8308	962	689	689
query69	484	340	310	310
query70	1065	933	944	933
query71	382	296	274	274
query72	5689	3281	3483	3281
query73	748	694	296	296
query74	8938	8840	8523	8523
query75	3055	2813	2431	2431
query76	3545	1056	626	626
query77	826	351	269	269
query78	9868	9988	9262	9262
query79	1209	905	577	577
query80	618	575	485	485
query81	532	258	227	227
query82	205	146	112	112
query83	265	247	234	234
query84	258	125	103	103
query85	866	508	441	441
query86	390	319	296	296
query87	2794	2824	2786	2786
query88	3099	2214	2211	2211
query89	383	358	323	323
query90	2159	147	150	147
query91	170	157	142	142
query92	78	66	63	63
query93	1718	916	528	528
query94	551	322	290	290
query95	569	369	299	299
query96	573	451	202	202
query97	2312	2354	2302	2302
query98	229	208	197	197
query99	580	566	480	480
Total cold run time: 257373 ms
Total hot run time: 173782 ms

@doris-robot
Copy link

doris-robot commented Jan 8, 2026

ClickBench: Total hot run time: 27.08 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18a288a56cbbd910484e4888104c88c70be140e4, data reload: false

query1	0.06	0.05	0.05
query2	0.16	0.05	0.04
query3	0.26	0.08	0.08
query4	1.61	0.12	0.11
query5	0.27	0.26	0.26
query6	1.16	0.68	0.65
query7	0.03	0.02	0.03
query8	0.05	0.04	0.04
query9	0.59	0.50	0.49
query10	0.54	0.54	0.55
query11	0.17	0.10	0.09
query12	0.15	0.11	0.11
query13	0.60	0.59	0.57
query14	0.96	0.94	0.95
query15	0.79	0.77	0.78
query16	0.40	0.40	0.41
query17	1.02	1.05	1.01
query18	0.23	0.21	0.22
query19	1.90	1.79	1.86
query20	0.02	0.01	0.02
query21	15.48	0.27	0.14
query22	5.21	0.05	0.04
query23	15.77	0.28	0.10
query24	1.95	0.69	0.60
query25	0.07	0.05	0.08
query26	0.13	0.13	0.14
query27	0.08	0.06	0.06
query28	3.92	1.06	0.88
query29	12.61	3.90	3.15
query30	0.29	0.14	0.12
query31	2.84	0.68	0.39
query32	3.23	0.57	0.46
query33	3.03	3.01	3.04
query34	16.70	5.14	4.44
query35	4.44	4.50	4.50
query36	0.65	0.50	0.50
query37	0.26	0.07	0.06
query38	0.07	0.04	0.03
query39	0.06	0.03	0.03
query40	0.18	0.14	0.13
query41	0.09	0.04	0.03
query42	0.08	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 98.15 s
Total hot run time: 27.08 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 8, 2026

FE Regression Coverage Report

Increment line coverage 89.66% (52/58) 🎉
Increment coverage report
Complete coverage report

2 similar comments
@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 8, 2026

FE Regression Coverage Report

Increment line coverage 89.66% (52/58) 🎉
Increment coverage report
Complete coverage report

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 8, 2026

FE Regression Coverage Report

Increment line coverage 89.66% (52/58) 🎉
Increment coverage report
Complete coverage report

@morrySnow
Copy link
Contributor Author

morrySnow commented Jan 8, 2026

run p0 5

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 8, 2026

FE Regression Coverage Report

Increment line coverage 89.66% (52/58) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@morrySnow @Thearas @doris-robot @hello-stephen