-
Notifications
You must be signed in to change notification settings - Fork 840
Description
Is your feature request related to a problem? Please describe.
#7046 introduces a new mode for store gateway to query parquet files intead of TSDB blocks. However, for production environment where store gateway still queries TSDB blocks, it is not easy to migrate to parquet mode without down time.
Besides, Parquet files are converted from TSDB blocks and it takes longer time for them to be available. Before parquet files are converted, to make latest data queryable Store Gateway might still need to support querying TSDB blocks.
Describe the solution you'd like
I want to propose a hybrid mode of Store Gateway that is able to query both parquet files and TSDB blocks. Hybrid Store Gateway still needs to sync TSDB blocks at start up and periodically but it only needs to sync most recent blocks for a configurable time period (such as recent X days). For the blocks to query asked from Querier, if a block is available as parquet file then parquet file is queried otherwise, TSDB block is queried. This hybrid mode allows smooth migration from the TSDB store gateway to full parquet gateway eventually.
This is a very similar approach to what we did for parquet querier. If a block is available as parquet file then querier queries parquet file otherwise it queries Store Gateway for TSDB blocks.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.