PREFECT_TASKS_DEFAULT_PERSIST_RESULT vs PREFECT_TASKS_DEFAULT_NO_CACHE

[gsakkis]

What is the difference between these two settings? More specifically PREFECT_TASKS_DEFAULT_PERSIST_RESULT=False seems to have the same effect with PREFECT_TASKS_DEFAULT_NO_CACHE=True; in both cases task results are not cached. When I would use the one vs the other?

[bdalpe]

Task caching and result persistence are intimately related. Because task caching relies on loading a known result, task caching will only work when your task can persist its output to a fixed and known location.

Therefore any configuration which explicitly avoids result persistence will result in your task never using a cache, for example setting persist_result=False.

https://docs.prefect.io/v3/concepts/caching#cache-keys

What each setting does

PREFECT_TASKS_DEFAULT_PERSIST_RESULT=False

• Controls result persistence: whether task return values are written to storage (e.g. ~/.prefect/storage/).
• False = “by default, do not persist task results.”
• Caching depends on persisted results, so turning off persistence also means no caching.

PREFECT_TASKS_DEFAULT_NO_CACHE=True

• Controls the default cache policy: sets the default to the NO_CACHE policy.
• From the caching docs: the NO_CACHE policy “always returns None” for the cache key and “avoids caching and result persistence altogether.”
• So this explicitly disables both cache lookup and persistence for tasks using that default.

So both can lead to “no caching and no persistence,” but they do it via different knobs.

Difference in intent

Setting	What you’re configuring	Main intent
PREFECT_TASKS_DEFAULT_PERSIST_RESULT=False	Whether results are stored at all	“Don’t persist results” (storage/cost/privacy). No cache as a consequence.
PREFECT_TASKS_DEFAULT_NO_CACHE=True	Default cache policy	“Don’t use task caching” (always re-run). The NO_CACHE policy also disables persistence for those tasks.

So:

• PREFECT_TASKS_DEFAULT_PERSIST_RESULT=False
  Use when your goal is no result persistence (e.g. save storage, avoid storing large or sensitive data, or you don’t need results after the run). Caching is disabled because there’s nothing to cache.

• PREFECT_TASKS_DEFAULT_NO_CACHE=True
  Use when your goal is no caching (every task run should execute, no reuse of previous results). You’re saying “default to NO_CACHE”; that policy then also avoids persisting results for those tasks.

When to use which

• If you only care that task results are not cached, either setting can achieve that; they overlap in effect.
• Prefer PREFECT_TASKS_DEFAULT_PERSIST_RESULT=False when you’re thinking in terms of storage/persistence (“don’t store results”).
• Prefer PREFECT_TASKS_DEFAULT_NO_CACHE=True when you’re thinking in terms of caching (“don’t reuse previous runs; always run the task”).

There’s also PREFECT_TASKS_DISABLE_CACHING: when True, it “disables caching on all tasks regardless of cache policy.” That’s a global override for caching only, without changing the default persist/cache policy.

[gsakkis]

Thanks for the prompt reply! One clarification regarding PREFECT_TASKS_DEFAULT_PERSIST_RESULT:

False = “by default, do not persist task results.”

I expected that in order to override this default, a task would need to pass persist_result=True. Apparently though task results are also persisted by passing an explicit cache policy (e.g. @task(cache_policy=DEFAULT)). Is this expected behavior?

[bdalpe]

Yes. That behavior is expected given how the code is written.

What happens is:

1. In Task.__init__ (tasks.py):

The global default PREFECT_TASKS_DEFAULT_PERSIST_RESULT is not read when the task is defined. Only the decorator args and inference from other options are used:

        # result persistence settings
        self._user_persist_result = persist_result
        if persist_result is None:
            if any(
                [
                    cache_policy
                    and cache_policy != NO_CACHE
                    and cache_policy != NotSet,
                    cache_key_fn is not None,
                    result_storage_key is not None,
                    result_storage is not None,
                    result_serializer is not None,
                ]
            ):
                persist_result = True

So when you do @task(cache_policy=DEFAULT) and don’t pass persist_result:

• persist_result is None.
• cache_policy is DEFAULT (not NO_CACHE, not NotSet), so the condition is true.
• The code sets persist_result = True and then self.persist_result = True.

So the task ends up with an explicit persist_result=True on the task object, inferred from the cache policy.

2. At runtime (task_engine.py):

Persistence is chosen in this order:

            if self.task.persist_result is not None:
                persist_result = self.task.persist_result
            elif settings.tasks.default_persist_result is not None:
                persist_result = settings.tasks.default_persist_result
            else:
                persist_result = should_persist_result()

Because the task already has self.task.persist_result = True (from the inference above), the first branch is used and the global default is never consulted for that task.

So:

• Yes, it’s expected: the design is “if you enable caching (or other result-related options) and don’t say otherwise, we set persist_result=True so caching/storage can work.”
• Overriding the global default happens in two ways:
  1. Explicit: @task(persist_result=True) (or False).
  2. Implicit: passing any of cache_policy (other than NO_CACHE), cache_key_fn, result_storage_key, result_storage, or result_serializer causes persist_result to be inferred as True when you don’t pass persist_result.

If you want “no persistence” to win even when a cache policy is set, you have to set it explicitly: @task(cache_policy=DEFAULT, persist_result=False). The code will then log that it’s ignoring the cache policy for that task (see the branch around 543–547 in tasks.py).