Skip to content

Conversation

@scotts
Copy link
Contributor

@scotts scotts commented Nov 9, 2024

I'm trying to make our workflow triggering smarter. For example, when we're making doc changes, we're still triggering Mac and CUDA wheels, and those jobs take a long time and we just ignore them anyway. I'm less concerned about wasting resources, and more concerned about making sure we see important signals. All workflows have a self reference. In addition to that, my reasoning on why triggering what when:

build_ffmpeg.yaml
Only triggered when the script itself is updated. The script has no dependence on our code.

cpp_tests.yaml
Triggered when anything under src/torchcodec/* or test/* is changed. That means Python only changes will also trigger these tests, but I think trying to differentiate at that level is going to be too subtle and error prone. Doc changes do not trigger the C++ tests.

docs.yaml
Triggered when anything under src/torchcodec/* or docs/* is changed. Our source code has docs, and obviously we build the docs when docs files change. Notably we don't need to trigger the docs when tests change.

linux_wheel.yaml, linux_cuda_wheel.yaml, macos_wheel.yaml
All are triggered when anything under src/torchcodec/*, test/* or packaging/* is changed. These are our main test drivers, so we want these to run when all source code, tests or packaging scripts change. Doc changes do not need to trigger wheel builds and tests.

reference_resources.yaml
Only triggered when the script itself is updated. The script has no dependence on our code.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label Nov 9, 2024
@scotts scotts marked this pull request as ready for review November 9, 2024 03:18
@scotts scotts force-pushed the workflow_partition branch from 906dbf2 to 17a3ff5 Compare November 9, 2024 18:34
Copy link
Contributor

@NicolasHug NicolasHug left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @scotts , this looks good, happy to try this out. The best way to test this probably is to merge the PR and pay extra attention to the CI signals within the next few days.

Doc changes do not need to trigger wheel builds

Note that this may soon be the case, depending on how we go with #335, but this PR can be merged first nonetheless.

@scotts scotts merged commit 10be66b into meta-pytorch:main Nov 12, 2024
56 of 59 checks passed
@scotts scotts deleted the workflow_partition branch November 12, 2024 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants