Built-in way of running only on changed files

[maxammann]

In default configuration zizmor will scan all the files in a repository. The GitHub annotations will also highlight issues in files other than the changed ones. This would then be shown as:

Is there a build-in way to only scan files that changed?

[woodruffw]

Hey @maxammann, good to see you again 🙂

Is there a build-in way to only scan files that changed?

Not within zizmor, no -- zizmor itself isn't aware of git repositories or git state, so it doesn't have the ability to track changes over time. That was a semi-intentional decision on my part, since (1) zizmor is architected to be fast on re-runs, and (2) I wanted people to be able to run it over single files (or directories containing multiple repos) just like the normal git repo case.

With that said, you can approximate this in a few ways: if you're using pre-commit it'll do this filtering automatically for you. For zizmor-action you could change your triggers on the workflow itself to include a paths: filter, e.g.:

on:
  pull_request:
    paths:
      - '.github/workflows/*.yml'

...which will tell GitHub Actions to only dispatch the workflow if a changed path matches. Keep in mind though that you'll need to also track any composite action paths, Dependabot paths, etc. if you use them.

Ref: https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#pull_request

More generally however I'd recommend avoiding GitHub annotations (if you can) -- they have tons of edge cases (like this one) and are generally buggy in comparison to either the "plain" output mode or SARIF via GitHub's "advanced security" feature. Some caveats are documented here:

https://docs.zizmor.sh/usage/#github-annotations

https://github.com/zizmorcore/zizmor-action#annotations

(I also need to update those caveats to mention that GitHub will sometimes disappear annotations if they include the final line of a given file.)

[maxammann]

good to see you again 🙂

likewise :)

which will tell GitHub Actions to only dispatch the workflow if a changed path matches.

Oh I mean something different. Just run it on changed files. I still want to run it for every commit, but it should only scan files that have been touched in the current branch (compared to the default branch). This would definitely be something that is added in zizmor-action as zizmor should not be aware of git.

avoiding GitHub annotations

Yeah its pretty annoying. We dont use advanced security so I use annotation in "best effort" mode but also gather all findings centrally in DefectDojo.
Wasnt aware of the "final line caveat". That is annoying!

[woodruffw]

I still want to run it for every commit, but it should only scan files that have been touched in the current branch (compared to the default branch). This would definitely be something that is added in zizmor-action as zizmor should not be aware of git.

Oh gotcha, I see what you mean. Yeah, that sounds like a useful thing the action could do, although I'm kind of wary of making it the default (since I think the default of scanning everything is useful/I personally like it 😅).

WDYT about something like a changes-only: true or similar input?

Yeah its pretty annoying. We dont use advanced security so I use annotation in "best effort" mode but also gather all findings centrally in DefectDojo.
Wasnt aware of the "final line caveat". That is annoying!

Oh interesting, TIL about DefectDojo. Do they consume SARIF?

Another thing I've been thinking about for this action is a "comment" mode, where zizmor-action would leave a comment on your PR (or update an existing one) with the render from the "plain" mode. That might be useful in these contexts, since it won't be as buggy as annotations. OTOH it'll be a bit noisier/less cleanly integrated into the GH platform.

[maxammann]

WDYT about something like a changes-only: true or similar input?

that sounds perfect.

Oh interesting, TIL about DefectDojo. Do they consume SARIF?

yeah exactly. the main point of the tool is to have various importers that normalize tool output.

comment on your PR

Potentially it could only do that if it knows that GH will drop annoations. Anyways good idea, but big potential for too much noise imo, especially a Zizmor is one tool out of many.