[opt](cloud) Prioritize scheduling the most recently active tablets in cloud #59539

deardeng · 2026-01-04T11:19:52Z

What problem does this PR solve?

Starting from the getBackendId of the replica, implement a sliding window approach (in TabletAccessStats.java) to count read and write operations over the last hour. The accuracy of this count isn't critical, as the aim is to indicate relative activity levels.
Display these statistics in the SHOW TABLETS FROM TABLE and SHOW TABLET {tabletId} commands.

show tablet 1767773242851\G
***************************[ 1. row ]***************************
...
AccessCount1H  | 1107
LastAccessTime | 1767782384995
...

Integrate with the existing balancing strategy:

For each scheduling operation, identify active databases, tables, and partitions from the active tablets. Schedule operations first on the most active databases, tables, and partitions.
When selecting tablets from the source BE, modify the previous random selection to prioritize non-active tablets for scheduling, keeping active tablets as stable as possible on the source.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-01-04T11:19:57Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

deardeng · 2026-01-08T10:01:09Z

run buildall

deardeng · 2026-01-08T10:27:22Z

run buildall

doris-robot · 2026-01-08T10:47:09Z

TPC-H: Total hot run time: 31449 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 51e3d4ba0880d99c10399578a6c5f5d5b98f03a3, data reload: false

------ Round 1 ----------------------------------
q1	17641	4341	4076	4076
q2	2073	384	241	241
q3	10096	1289	707	707
q4	10209	864	326	326
q5	7495	2109	1880	1880
q6	201	175	139	139
q7	944	792	671	671
q8	9286	1413	1149	1149
q9	4855	4462	4552	4462
q10	6729	1825	1431	1431
q11	514	310	288	288
q12	693	726	607	607
q13	17776	3796	3106	3106
q14	295	301	284	284
q15	585	514	504	504
q16	665	666	624	624
q17	663	842	452	452
q18	6580	6521	6446	6446
q19	943	965	610	610
q20	394	358	250	250
q21	3004	2484	2222	2222
q22	1020	1032	974	974
Total cold run time: 102661 ms
Total hot run time: 31449 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4107	4058	4045	4045
q2	338	390	315	315
q3	2130	2581	2242	2242
q4	1342	1743	1312	1312
q5	4114	3977	4048	3977
q6	215	170	134	134
q7	1893	1825	1675	1675
q8	2833	2547	2533	2533
q9	7289	7272	7297	7272
q10	2773	2786	2318	2318
q11	560	511	456	456
q12	748	792	659	659
q13	3684	4036	3376	3376
q14	300	305	311	305
q15	562	498	523	498
q16	646	737	640	640
q17	1136	1325	1379	1325
q18	8275	7748	8011	7748
q19	831	836	882	836
q20	1998	2107	1932	1932
q21	4780	4607	4392	4392
q22	1200	1088	1047	1047
Total cold run time: 51754 ms
Total hot run time: 49037 ms

doris-robot · 2026-01-08T10:58:11Z

TPC-DS: Total hot run time: 176500 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 51e3d4ba0880d99c10399578a6c5f5d5b98f03a3, data reload: false

query5	4562	654	491	491
query6	359	260	239	239
query7	4227	478	286	286
query8	335	270	239	239
query9	8794	2676	2690	2676
query10	490	397	348	348
query11	15112	15207	14956	14956
query12	191	127	121	121
query13	1272	515	414	414
query14	7851	3075	2788	2788
query14_1	2728	2685	2735	2685
query15	195	191	182	182
query16	991	498	478	478
query17	1372	711	600	600
query18	2715	436	363	363
query19	241	238	207	207
query20	140	123	118	118
query21	221	150	134	134
query22	3963	4034	3851	3851
query23	17008	16268	15790	15790
query23_1	15955	16038	15814	15814
query24	7146	1622	1223	1223
query24_1	1247	1295	1267	1267
query25	606	503	444	444
query26	1249	271	162	162
query27	2732	479	303	303
query28	4452	2219	2181	2181
query29	799	594	512	512
query30	305	244	207	207
query31	812	650	581	581
query32	80	73	64	64
query33	538	345	300	300
query34	925	953	577	577
query35	744	759	701	701
query36	928	914	885	885
query37	133	93	79	79
query38	2770	2766	2778	2766
query39	767	741	717	717
query39_1	699	720	711	711
query40	224	141	140	140
query41	68	62	63	62
query42	104	100	99	99
query43	489	471	446	446
query44	1414	785	789	785
query45	191	186	185	185
query46	899	1042	650	650
query47	1514	1503	1430	1430
query48	333	367	271	271
query49	613	456	343	343
query50	708	313	214	214
query51	3841	3803	3828	3803
query52	122	112	105	105
query53	312	348	287	287
query54	316	280	256	256
query55	83	86	75	75
query56	312	346	311	311
query57	1076	1019	993	993
query58	265	280	266	266
query59	2162	2254	2226	2226
query60	345	367	317	317
query61	199	183	178	178
query62	406	358	323	323
query63	320	281	282	281
query64	5007	1340	1042	1042
query65	3784	3799	3792	3792
query66	1360	411	315	315
query67	15109	15182	15248	15182
query68	6251	1048	744	744
query69	518	364	347	347
query70	1125	1027	1049	1027
query71	390	314	301	301
query72	6074	3368	3433	3368
query73	794	764	333	333
query74	8861	8907	8571	8571
query75	2810	2830	2435	2435
query76	3930	1128	692	692
query77	513	407	296	296
query78	9715	9690	9127	9127
query79	1595	987	634	634
query80	668	634	532	532
query81	531	285	248	248
query82	248	158	121	121
query83	312	286	258	258
query84	266	140	115	115
query85	1011	617	571	571
query86	396	343	324	324
query87	2958	3019	2856	2856
query88	4353	2290	2305	2290
query89	413	369	354	354
query90	2202	167	161	161
query91	196	186	160	160
query92	86	79	74	74
query93	1846	1021	565	565
query94	590	375	275	275
query95	601	417	324	324
query96	657	538	232	232
query97	2389	2395	2328	2328
query98	251	242	228	228
query99	618	584	508	508
Total cold run time: 259275 ms
Total hot run time: 176500 ms

fe/fe-core/src/main/java/org/apache/doris/common/util/SlidingWindowAccessStats.java

fe/fe-core/src/main/java/org/apache/doris/catalog/TabletAccessStats.java

fe/fe-core/src/main/java/org/apache/doris/catalog/TabletInvertedIndex.java

fe/fe-core/src/main/java/org/apache/doris/common/util/SlidingWindowAccessStats.java

fe/fe-core/src/main/java/org/apache/doris/catalog/TabletAccessStats.java

…n cloud

deardeng requested review from dataroaring, gavinchou and w41ter as code owners January 4, 2026 11:19

deardeng marked this pull request as draft January 4, 2026 11:20

deardeng force-pushed the active-tablet branch 2 times, most recently from ac383bb to a71331b Compare January 8, 2026 09:58

deardeng marked this pull request as ready for review January 8, 2026 10:01

gavinchou reviewed Jan 9, 2026

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/common/util/SlidingWindowAccessStats.java Show resolved Hide resolved

gavinchou reviewed Jan 9, 2026

View reviewed changes

fe/fe-core/src/main/java/org/apache/doris/common/util/SlidingWindowAccessStats.java Show resolved Hide resolved