Pre Aggregations are not getting refreshed

[ananya-level]

Describe the bug
Pre-aggregations are not updated if I want to build the pre-aggregations in the source table with an external false flag. [Snowflake in my case] I also do not have refresh worker enabled, and the condition for refresh is default, every 1 hour without any SQL query for it.

It creates aggregation if it does not exist correctly, but then fails to refresh the older preaggregation. I have set the configuration to build in the background and serve the data from the existing one, but upon subsequent queries, it fails to serve the data. I do not want to set renewQuery to true for all the requests. There is no error, and the data is stale.

To Reproduce
Steps to reproduce the behavior:

1. Go to Cube playground and do a query that uses pre-aggregation.
2. Observe refresh_key interval defined for pre-aggregation. (set it to some lower value to verify the issue quickly)
3. Observe the preaggregation used in the first query.
4. Make a new query by changing the time range of the query after the defined refresh_key interval
5. Observe that the response is still from the older preaggregation.

Screenshots
Attached section of code where we see the issue is

Version:
1.3.9

Additional context
At line 220, for us both, the flags are false, which makes the if clause true. Now, because there is a return statement, it never schedules a new refresh for the pre-aggregation

[igorlukanin]

I also do not have refresh worker enabled

Why is that? I'd recommend to set up and use a refresh worker to build and refresh pre-aggregations in background. This is the recommended configuration.

[ananya-level]

I also do not have refresh worker enabled

Why is that? I'd recommend to set up and use a refresh worker to build and refresh pre-aggregations in background. This is the recommended configuration.

We want it to be on demand because we have a lot of tables and customer's usage is sporadic. Setting up a dedicated refresh worker that builds every hour was costly and most of them were not even used.

[igorlukanin]

OK, I get it. The current implementation will renew the pre-aggregation in background while serving the API request from the existing pre-aggregation (with possibly stale data). The reason is that waiting for the pre-aggregation to build takes seconds, sometimes minutes, and it's better to respond quickly than make the user wait too long.

If you're using the REST API, you can make Cube renew the pre-aggregation in the foreground by passing the renewQuery flag (set it to true).

However, honestly, if your usage is sporadic, then you'd probably be better off letting users query the database directly.