Add ability to show which projects would hit cache

[rlmestre]

• I'd be willing to implement this feature (contributing guide)

Description

When using nx run-many or similar commands, there's no way to predetermine which runs would be retrieved from the cache and which ones wouldn't without running them.

Motivation

Similar to why nx show projects --affected (previously nx print-affected --select=projects) exists: knowing which outputs need recomputation in advance may be useful for control flow or other reasons.

I've thought about this to avoid stalling git hooks for reasons that aren't git-related. We have a pre-push hook that checks if linting passes for affected projects. When many projects need recomputation it takes a while which, even though engineers are aware of why it does, it's not ideal.

Checking if the targets are in the cache would let us short-circuit the process. We can have the git hook only check if all runs are in the cache and otherwise fail so the developer runs the process separately and have the cache rebuilt.

Suggested Implementation

A flag for nx show similar to -affected would probably be ideal:
nx show projects --cached --target=test

Alternate Implementations

None

Related

#6681 (closed as stale)

[mayteio]

+1 on this. I'd love to skip some code-generation tasks in CI if I know ahead of time that all tasks will hit the cache.

[LuminusDev]

+1

[FrozenPandaz]

The reality is that there is no way to predetermine this. Nx waits until the last minute, right before running the task, to determine if it can reuse the cache. This is tasks can depend on the outputs of other tasks which would be run before it. So without running tasks, we don't know for certain if something is able to pull from cache.

[markwhitfeld]

Just to add a potential use case here, and the reason why I searched for this.
I would love to make my CI as efficient as possible, so I would like to determine if there are any tasks that will need to be run BEFORE I do my npm install.
My thought process here is that my CI would:

• run npm i -g nx to install nx globally
• run something like nx build my-project --assert-cached, where --assert-cached is a new option in nx cli
  • this would run successfully if the target task and their dependencies are cached and do not require to be run
  • this would fail if the target task or any dependencies do not match the cache and require a run
• If the above step succeeds, then I can be done with my CI run
• If it fails, then I install npm deps and run the tasks the normal way

How can this work without running an npm install first?

• As far as I understand with nx caching, it is purely based off of the hash of the inputs.
  • As far as my mental model works, this can be done without actually executing the task.
  • I am assuming that the package.json and package lock are inputs, and not any files in the node_modules folder. This means that all inputs are actually available to be able to calculate the hash for, without the need for the node_modules to be present.
  • So, if the cache matches, then we download the cache, write the necessary output, and now we have the inputs required to assess the hash of any dependant tasks. In this way, a task and the tasks it depends on could all be checked.

What about plugins and inferred tasks?

• Unfortunately inferred tasks would need to be installed first so that the project graph can infer the target.
• Non-inferred tasks (explicitly defined in the project.json, etc.) can use this technique without npm install because they can be added to the graph without inference and their cache can be assessed.

Does this make sense?
Are my assumptions correct, and therefore is this feasible?