Skip to content

[Fix](Variant) Fix variant compaction empty path handling #59544

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
eldenmoon merged 1 commit into from
Jan 5, 2026
Merged

[Fix](Variant) Fix variant compaction empty path handling #59544

eldenmoon merged 1 commit into from
Jan 5, 2026

Conversation

@eldenmoon
Copy link
Member

@eldenmoon eldenmoon commented Jan 4, 2026

  • Skip VARIANT root node when building sparse merge readers

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@eldenmoon
Fix variant compaction empty path handling
c359139 
- Skip VARIANT root node when building sparse merge readers

- Add regression test for empty key path during compaction
@Thearas
Copy link
Contributor

Thearas commented Jan 4, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@eldenmoon eldenmoon added dev/4.0.x dev/3.1.x usercase Important user case type label labels Jan 4, 2026
@eldenmoon
Copy link
Member Author

eldenmoon commented Jan 4, 2026

run buildall

@doris-robot
Copy link

doris-robot commented Jan 4, 2026

TPC-H: Total hot run time: 31549 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c359139719f3ee3a4df24e37843604d455c1f1f9, data reload: false

------ Round 1 ----------------------------------
q1	17629	4260	4073	4073
q2	2046	357	243	243
q3	10145	1294	725	725
q4	10282	876	329	329
q5	7522	2006	1931	1931
q6	213	173	136	136
q7	908	774	651	651
q8	9262	1410	1174	1174
q9	4773	4557	4587	4557
q10	6735	1779	1419	1419
q11	536	312	294	294
q12	695	738	562	562
q13	17804	3828	3125	3125
q14	285	293	276	276
q15	574	512	506	506
q16	681	661	628	628
q17	671	823	506	506
q18	6617	6384	6390	6384
q19	1096	962	588	588
q20	400	387	250	250
q21	3083	2392	2241	2241
q22	1024	1041	951	951
Total cold run time: 102981 ms
Total hot run time: 31549 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4055	4043	4038	4038
q2	330	403	334	334
q3	2077	2632	2175	2175
q4	1331	1746	1300	1300
q5	4143	4055	4084	4055
q6	215	181	137	137
q7	1839	1803	1667	1667
q8	2840	2517	2441	2441
q9	7065	7180	7133	7133
q10	2695	2715	2255	2255
q11	564	469	452	452
q12	713	802	653	653
q13	3581	4071	3386	3386
q14	389	415	295	295
q15	540	504	532	504
q16	639	679	615	615
q17	1139	1226	1324	1226
q18	8108	7714	7949	7714
q19	912	931	908	908
q20	2047	2030	1928	1928
q21	4833	4595	4499	4499
q22	1176	1106	1052	1052
Total cold run time: 51231 ms
Total hot run time: 48767 ms

@doris-robot
Copy link

doris-robot commented Jan 4, 2026

TPC-DS: Total hot run time: 173554 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c359139719f3ee3a4df24e37843604d455c1f1f9, data reload: false

query5	5816	577	439	439
query6	345	219	207	207
query7	4241	449	266	266
query8	359	250	251	250
query9	8776	2619	2591	2591
query10	558	369	311	311
query11	15208	15097	16142	15097
query12	205	122	111	111
query13	1367	551	400	400
query14	7850	3001	2824	2824
query14_1	2792	2773	2740	2740
query15	268	213	178	178
query16	1016	542	468	468
query17	1312	750	618	618
query18	2742	439	334	334
query19	304	223	204	204
query20	120	113	113	113
query21	211	132	114	114
query22	4232	4242	4291	4242
query23	16154	15643	15396	15396
query23_1	15486	15523	15401	15401
query24	6951	1537	1143	1143
query24_1	1181	1172	1190	1172
query25	599	448	381	381
query26	1207	257	143	143
query27	2711	430	290	290
query28	4525	2130	2120	2120
query29	752	513	413	413
query30	306	242	193	193
query31	741	639	545	545
query32	74	62	61	61
query33	517	310	277	277
query34	850	880	511	511
query35	753	767	699	699
query36	816	887	843	843
query37	117	84	72	72
query38	2643	2645	2673	2645
query39	770	748	725	725
query39_1	716	705	702	702
query40	206	125	110	110
query41	63	59	58	58
query42	100	105	103	103
query43	486	465	420	420
query44	1282	769	725	725
query45	185	179	174	174
query46	841	956	590	590
query47	1456	1519	1323	1323
query48	330	317	242	242
query49	600	409	318	318
query50	641	268	203	203
query51	3807	3981	3751	3751
query52	100	107	97	97
query53	297	318	277	277
query54	279	260	242	242
query55	75	75	66	66
query56	278	303	287	287
query57	1018	1054	968	968
query58	273	254	249	249
query59	2122	2194	2056	2056
query60	317	315	294	294
query61	168	159	174	159
query62	397	339	336	336
query63	292	267	268	267
query64	4857	1311	1021	1021
query65	3831	3688	3702	3688
query66	1355	398	297	297
query67	15396	15849	15119	15119
query68	8518	978	713	713
query69	502	355	308	308
query70	1041	945	928	928
query71	385	300	281	281
query72	5997	3425	3486	3425
query73	776	720	295	295
query74	8855	8828	8538	8538
query75	2796	2803	2433	2433
query76	3749	1049	635	635
query77	520	358	276	276
query78	9705	9884	9132	9132
query79	1484	860	584	584
query80	615	541	472	472
query81	511	265	224	224
query82	207	140	110	110
query83	266	260	233	233
query84	260	121	100	100
query85	881	520	458	458
query86	347	289	320	289
query87	2846	2858	2775	2775
query88	3603	2198	2176	2176
query89	391	340	326	326
query90	2300	147	145	145
query91	190	163	138	138
query92	90	62	66	62
query93	1844	921	541	541
query94	574	327	302	302
query95	543	370	296	296
query96	573	486	208	208
query97	2317	2355	2285	2285
query98	239	197	193	193
query99	578	581	519	519
Total cold run time: 259003 ms
Total hot run time: 173554 ms

@doris-robot
Copy link

doris-robot commented Jan 4, 2026

ClickBench: Total hot run time: 26.73 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c359139719f3ee3a4df24e37843604d455c1f1f9, data reload: false

query1	0.05	0.04	0.06
query2	0.11	0.05	0.04
query3	0.25	0.09	0.08
query4	1.62	0.11	0.12
query5	0.28	0.27	0.25
query6	1.14	0.66	0.65
query7	0.03	0.03	0.02
query8	0.05	0.04	0.05
query9	0.56	0.50	0.52
query10	0.58	0.55	0.55
query11	0.16	0.09	0.09
query12	0.15	0.11	0.12
query13	0.61	0.59	0.58
query14	0.94	0.96	0.94
query15	0.78	0.77	0.78
query16	0.39	0.43	0.40
query17	1.07	1.05	1.04
query18	0.23	0.22	0.21
query19	1.90	1.87	1.85
query20	0.02	0.02	0.02
query21	15.44	0.25	0.14
query22	5.14	0.05	0.05
query23	15.83	0.28	0.10
query24	1.49	0.29	0.18
query25	0.11	0.09	0.05
query26	0.15	0.15	0.14
query27	0.08	0.08	0.05
query28	4.42	1.06	0.87
query29	12.58	3.95	3.17
query30	0.28	0.14	0.12
query31	2.82	0.62	0.40
query32	3.26	0.57	0.46
query33	2.94	3.08	3.01
query34	16.75	5.04	4.44
query35	4.46	4.47	4.44
query36	0.65	0.51	0.48
query37	0.11	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.02
query40	0.16	0.14	0.13
query41	0.09	0.04	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 97.87 s
Total hot run time: 26.73 s

@doris-robot
Copy link

doris-robot commented Jan 4, 2026

BE UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 53.26% (18965/35605)
Line Coverage 39.22% (175948/448576)
Region Coverage 33.76% (136137/403236)
Branch Coverage 34.72% (58785/169306)

@hello-stephen
Copy link
Contributor

hello-stephen commented Jan 4, 2026

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 72.23% (25145/34810)
Line Coverage 58.96% (263795/447394)
Region Coverage 53.73% (218911/407423)
Branch Coverage 55.32% (93965/169872)

csun5285
csun5285 approved these changes Jan 5, 2026
View reviewed changes
Copy link
Contributor

@csun5285 csun5285 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Jan 5, 2026

PR approved by anyone and no changes requested.

airborne12
airborne12 approved these changes Jan 5, 2026
View reviewed changes
Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 5, 2026
@github-actions
Copy link
Contributor

github-actions bot commented Jan 5, 2026

PR approved by at least one committer and no changes requested.

@eldenmoon eldenmoon merged commit fa40415 into apache:master Jan 5, 2026
32 of 33 checks passed
github-actions bot pushed a commit that referenced this pull request Jan 5, 2026
@eldenmoon
[Fix](Variant) Fix variant compaction empty path handling (#59544)
3083572 
- Skip VARIANT root node when building sparse merge readers
@github-actions github-actions bot mentioned this pull request Jan 5, 2026
Open
github-actions bot pushed a commit that referenced this pull request Jan 5, 2026
@eldenmoon
[Fix](Variant) Fix variant compaction empty path handling (#59544)
0d9ef7f 
- Skip VARIANT root node when building sparse merge readers
@github-actions github-actions bot mentioned this pull request Jan 5, 2026
Merged
yiguolei pushed a commit that referenced this pull request Jan 8, 2026
@github-actions @eldenmoon
branch-4.0: [Fix](Variant) Fix variant compaction empty path handling #…
82fa1a1 
…59544 (#59550)

Cherry-picked from #59544

Co-authored-by: lihangyu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@airborne12 airborne12 airborne12 approved these changes

+1 more reviewer

@csun5285 csun5285 csun5285 approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/3.1.x dev/4.0.3-merged reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@eldenmoon @Thearas @doris-robot @hello-stephen @airborne12 @csun5285 @yiguolei