Make background tasks TabletWriteLogHistorySyncer and StatisticsMetaManager configurable

[stdpain]

Feature request

Is your feature request related to a problem? Please describe.
Currently, the background tasks TabletWriteLogHistorySyncer and StatisticsMetaManager are not configurable. This restricts flexibility in tuning StarRocks deployment based on different use cases or workloads.

Describe the solution you'd like
Introduce configuration options (e.g., via properties or flags) that allow users to adjust the behavior, scheduling, and resource usage for the background tasks TabletWriteLogHistorySyncer and StatisticsMetaManager.

Describe alternatives you've considered

• Manual code changes or recompiling with different parameters (not ideal for operators)
• Disabling one or both of these tasks entirely (not flexible)

Additional context
Making these background tasks configurable would help users optimize for resource usage or stability according to their deployment requirements.

[arvindKandpal-ksolves]

Hi @stdpain,

I have started working on this issue.

Frontend: I have successfully made StatisticsMetaManager configurable by introducing statistic_manager_run_interval_sec.

Backend: I added the config variable tablet_write_log_sync_interval_sec, but I am unable to locate the logic for TabletWriteLogHistorySyncer in the current codebase.

I searched for sync_tablet_write_log_history and TabletWriteLogHistorySyncer in be/src/storage/ (specifically storage_engine.cpp and tablet_manager.cpp) but couldn't find the thread loop to apply the configuration.

Could you please point me to the correct file or function where TabletWriteLogHistorySyncer is implemented ?

Thanks!

[stdpain]

@arvindKandpal-ksolves There is no need to modify the backend code. Simply make these parameters configurable.

starrocks/fe/fe-core/src/main/java/com/starrocks/lake/TabletWriteLogHistorySyncer.java

Line 81 in 1fd3265

super("TabletWriteLogHistorySyncer", 60 * 1000L); // 60s interval

starrocks/fe/fe-core/src/main/java/com/starrocks/statistic/StatisticsMetaManager.java

Line 83 in 1fd3265

super("statistics-meta-manager", 60L * 1000L);

[arvindKandpal-ksolves]

Hi @stdpain,

I have raised the PR for this issue. As per your suggestion, I have modified the Frontend (Java) code to make both TabletWriteLogHistorySyncer and StatisticsMetaManager configurable.

Please take a look when you have a moment. Thanks!