Skip to content

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Jan 5, 2026

What problem does this PR solve?

if slot a is operative slots, but it is only used in a filter node which is parent of olapScan, this slot will be materialized by materialization node when set topn_lazy_materialization_using_index = true;

  • materialization(materializedSlots=[a, b], lazy=[c])
  • ->topn(b)
  • ->filter(a=1, output=(rowid, a, b))
  • ->materializeOlapScan(rowid, lazy=[c], T[a,b,c])
  • =>
  • materialization(materializedSlots=[b], lazy=[a, c])
  • ->topn(b)
  • ->project(rowid, b)
  • ->filter(a=1, output=(rowid, a, b))
  • ->materializeOlapScan(rowid, lazy=[a,c], T[a,b,c])
    Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly englefly force-pushed the index-lazy1 branch 2 times, most recently from 17ac6e2 to 99e49d8 Compare January 6, 2026 02:43
@englefly
Copy link
Contributor Author

englefly commented Jan 6, 2026

run buildall

@englefly englefly marked this pull request as ready for review January 6, 2026 02:50
@doris-robot
Copy link

TPC-H: Total hot run time: 31865 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 9a0ac95575a2d58e8d66fdb7b8ab73babe91a1bc, data reload: false

------ Round 1 ----------------------------------
q1	17607	4294	4093	4093
q2	2041	352	240	240
q3	10141	1281	734	734
q4	10210	828	315	315
q5	7585	2054	1833	1833
q6	186	173	140	140
q7	917	776	658	658
q8	9261	1538	1101	1101
q9	4819	4571	4564	4564
q10	6772	1778	1396	1396
q11	541	307	268	268
q12	699	728	584	584
q13	17801	3760	3071	3071
q14	280	289	264	264
q15	577	530	498	498
q16	662	679	651	651
q17	652	783	543	543
q18	6466	6396	6945	6396
q19	1267	1091	658	658
q20	408	381	251	251
q21	3144	2557	2601	2557
q22	1120	1074	1050	1050
Total cold run time: 103156 ms
Total hot run time: 31865 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4286	5328	4253	4253
q2	326	414	325	325
q3	2321	2772	2364	2364
q4	1400	1884	1402	1402
q5	4543	4317	4435	4317
q6	218	183	133	133
q7	2188	1855	1747	1747
q8	2536	2330	2428	2330
q9	7129	7217	7098	7098
q10	2471	2736	2240	2240
q11	577	468	453	453
q12	725	743	602	602
q13	3599	3784	3120	3120
q14	262	308	271	271
q15	524	491	478	478
q16	599	650	605	605
q17	1075	1284	1350	1284
q18	7641	7056	7092	7056
q19	828	794	816	794
q20	1890	1946	1820	1820
q21	4557	4239	4120	4120
q22	1061	1049	987	987
Total cold run time: 50756 ms
Total hot run time: 47799 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 173020 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 9a0ac95575a2d58e8d66fdb7b8ab73babe91a1bc, data reload: false

query5	4476	567	452	452
query6	319	238	214	214
query7	4234	454	267	267
query8	337	249	243	243
query9	8780	2638	2638	2638
query10	518	372	325	325
query11	15141	15091	15037	15037
query12	187	115	113	113
query13	1268	488	382	382
query14	6282	3005	2702	2702
query14_1	2619	2554	2579	2554
query15	204	192	173	173
query16	995	438	432	432
query17	1076	648	542	542
query18	2555	432	358	358
query19	218	220	193	193
query20	121	116	112	112
query21	213	136	118	118
query22	4074	4030	4083	4030
query23	15828	15704	15307	15307
query23_1	15264	15555	15379	15379
query24	7418	1540	1148	1148
query24_1	1177	1171	1186	1171
query25	552	464	422	422
query26	1248	269	157	157
query27	2761	455	288	288
query28	4547	2139	2140	2139
query29	818	561	469	469
query30	303	245	213	213
query31	771	635	565	565
query32	78	74	71	71
query33	545	345	296	296
query34	909	882	513	513
query35	749	784	694	694
query36	899	867	855	855
query37	131	94	81	81
query38	2686	2652	2707	2652
query39	775	770	707	707
query39_1	703	718	725	718
query40	222	135	132	132
query41	70	72	66	66
query42	103	104	103	103
query43	487	450	424	424
query44	1304	724	734	724
query45	193	188	176	176
query46	845	942	585	585
query47	1395	1470	1397	1397
query48	318	339	260	260
query49	626	452	346	346
query50	646	306	204	204
query51	3770	3814	3746	3746
query52	105	107	98	98
query53	298	326	274	274
query54	290	275	258	258
query55	79	77	74	74
query56	299	312	310	310
query57	1044	1050	984	984
query58	278	259	259	259
query59	2150	2164	2021	2021
query60	346	349	302	302
query61	186	185	203	185
query62	390	356	301	301
query63	296	264	276	264
query64	4857	1316	985	985
query65	3751	3695	3710	3695
query66	1409	413	300	300
query67	15890	15318	15245	15245
query68	7396	967	706	706
query69	499	350	325	325
query70	1056	929	956	929
query71	376	308	276	276
query72	6035	3357	3452	3357
query73	762	716	309	309
query74	8885	8763	8603	8603
query75	2811	2808	2453	2453
query76	3911	1063	634	634
query77	531	374	277	277
query78	9703	9978	9217	9217
query79	1245	854	591	591
query80	608	572	466	466
query81	496	262	234	234
query82	215	144	108	108
query83	260	252	246	246
query84	255	120	108	108
query85	919	499	443	443
query86	346	326	317	317
query87	2839	2861	2820	2820
query88	3144	2233	2240	2233
query89	391	351	332	332
query90	2014	170	152	152
query91	174	174	137	137
query92	72	71	62	62
query93	957	908	539	539
query94	573	315	286	286
query95	582	330	355	330
query96	579	469	209	209
query97	2341	2385	2279	2279
query98	229	201	198	198
query99	593	567	505	505
Total cold run time: 254263 ms
Total hot run time: 173020 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 26.98 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 9a0ac95575a2d58e8d66fdb7b8ab73babe91a1bc, data reload: false

query1	0.05	0.05	0.05
query2	0.10	0.05	0.05
query3	0.25	0.09	0.08
query4	1.61	0.12	0.11
query5	0.27	0.27	0.27
query6	1.14	0.66	0.65
query7	0.03	0.03	0.03
query8	0.05	0.03	0.04
query9	0.58	0.50	0.51
query10	0.55	0.55	0.56
query11	0.14	0.09	0.10
query12	0.14	0.11	0.10
query13	0.61	0.58	0.59
query14	0.95	0.94	0.94
query15	0.79	0.77	0.77
query16	0.42	0.38	0.40
query17	1.04	1.01	1.04
query18	0.24	0.21	0.21
query19	1.92	1.81	1.88
query20	0.02	0.02	0.01
query21	15.43	0.28	0.14
query22	5.24	0.05	0.05
query23	15.95	0.29	0.11
query24	1.72	0.36	1.21
query25	0.12	0.11	0.05
query26	0.14	0.13	0.14
query27	0.06	0.06	0.05
query28	5.07	1.06	0.88
query29	12.65	3.93	3.15
query30	0.28	0.13	0.12
query31	2.82	0.65	0.39
query32	3.24	0.56	0.47
query33	3.02	3.07	3.05
query34	16.84	5.09	4.48
query35	4.50	4.49	4.51
query36	0.67	0.50	0.49
query37	0.11	0.07	0.07
query38	0.08	0.04	0.04
query39	0.04	0.02	0.02
query40	0.18	0.14	0.13
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.03
Total cold run time: 99.23 s
Total hot run time: 26.98 s

@englefly englefly changed the title using index topn lazy [feat](topn lazy materialize)using index topn lazy Jan 6, 2026
@hello-stephen
Copy link
Contributor

FE Regression Coverage Report

Increment line coverage 26.19% (11/42) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants