tasks with runInCI: only run locally

[chadrik]

I have a moon.yml file like this:

tasks:
    main:
        deps:
          - ci_only
        options:
          cache: false
    ci_only:
        command: "echo 'should not run locally'"
        options:
            runInCI: 'only'

When I call moon run project:main the ci_only task runs. I can't find any way to make it not run. The docs say "Only run in CI, and not locally, if affected by touched files", so I also tried various permutations of using affectedFiles and moon run --affected, such as:

tasks:
    main:
        deps:
          - ci_only
        options:
          cache: false
    ci_only:
        command: "echo 'should not run locally'"
        inputs:
            - 'bar.py'
        options:
            runInCI: 'only'
            affectedFiles: true

Still the ci_only task displays in the task graph every time, and it runs whenever bar.py is touched.

Using moon 1.41.8.

Thanks!

What am I misunderstanding?

[milesj]

@chadrik runInCI only applies to moon ci.

[chadrik]

That means that you need to manually aggregate all the inputs of every task in the deps list, and manually keep them up to date over time.

In other words, instead of having something simple like this:

tasks:
  sync:
    inputs:
      - pyproject.toml
      - '$UV_PYTHON'

  build:
    inputs:
      - src/**/*.py
    deps:
      - sync

  test:
    inputs:
      - tests/**/*.py
    deps:
      - build

I need to roll forward the inputs of each dependency:

tasks:
  sync:
    inputs:
      - pyproject.toml
      - '$UV_PYTHON'

  build:
    inputs:
      - src/**/*.py
      # inputs from sync:
      - pyproject.toml
      - '$UV_PYTHON'
    deps:
      - sync

  test:
    inputs:
      - tests/**/*.py
      # inputs from sync:
      - pyproject.toml
      - '$UV_PYTHON'
      # inputs from build:
      - src/**/*.py
    deps:
      - build

It's not very ergonomic, especially with many input patterns and deep and/or wide task graphs, and it creates a lot of opportunity for accidental drift. What do you see as the advantage of this approach?

Also, this is a change in behavior from v1, correct? It appears to be from my testing.

[milesj]

You don't need to do that, you just need to curate your targets list. I guess it depends what you are trying to do exactly.

If you want to run tests, then just moon run app:test, and that will run build and sync.

If you want to run all tasks affected in a project, you'll need to use moon check app instead.

[chadrik]

One of the things I want to do is enable passing the list of changed files to the task using options: affectedFiles. I have to turn on --affected to do that, but once I do that, my tasks stop running unless I aggregate all inputs, as describe above.

Here's what I'm looking for in simplest terms:

run a task and its dependents, skipping any that have no changed files, and for those that do, pass the changed input files to the task

So, to achieve this, my request is one of these two things:

1. support passing changed files to tasks (options: affectedFiles) when not using --affected
2. change the behavior of affected to look for all affected tasks upstream from a target (or provide an option to do)

On topic 2, it seems like the behavior of affected has changed between v1 and v2, so now would be the time to do it.

---

Revisiting the test example above in more detail:

• A user edits a pyproject.toml file that affects the tooling
• A user edits a source file
• they commit and push
• moon ci app:test runs --> nothing happens "No tasks affected by changed files. Unable to execute action pipeline."

Seems reasonable that the tests should run, no? Honestly, it seems like a bug that they don't. The only solution I see to this is to aggregate all the inputs of upstream tasks, unless this behavior changes. And just to be clear: all of the tasks run in moon v1. This is a major change in behavior for v2.

If you want to run all tasks affected in a project, you'll need to use moon check app instead.

moon check app casts too wide a net. It could run tests for each version of python, plus linting, and formatting. I want my CI pipelines to run a specific target, not all targets.

[chadrik]

1. support passing changed files to tasks (options: affectedFiles) when not using --affected
2. change the behavior of affected to look for all affected tasks upstream from a target (or provide an option to do)

I had an idea on how to solve this somewhat gracefully: expand on the idea of inferInputs to allow individual control of inferred sources, and add "deps" as an optional source.

e.g.

tasks:
  test:
    deps:
      - build
    options:
      inferInputs:
        command: true
        env: false
        deps: true

[milesj]

@chadrik

One of the things I want to do is enable passing the list of changed files to the task using options: affectedFiles. I have to turn on --affected to do that, but once I do that, my tasks stop running unless I aggregate all inputs, as describe above.

This is exactly why file groups exist, so that you can easily share these inputs.

In regards to your test task, I would highly suggest including sources as inputs, and not just have the test files. Inputs = "any of these that change should trigger the test", and modifying sources are just that. Sources are required by test files (they typically import them), so they should be inputs.

support passing changed files to tasks (options: affectedFiles) when not using --affected

We'll probably have tokens to access changed files at some point.

Seems reasonable that the tests should run, no? Honestly, it seems like a bug that they don't. The only solution I see to this is to aggregate all the inputs of upstream tasks, unless this behavior changes. And just to be clear: all of the tasks run in moon v1. This is a major change in behavior for v2.

This is how it worked in v1, and it's funny you bring this up, because I have people in other threads stating they absolutely do NOT want it to work this way, because CI ends up over-running what it needs to. I'll add a CLI flag to control this since there's no happy path.