Improve lintrunner job runtime #14240

swolchok · 2025-09-11T23:17:14Z

lintrunner is required to land, but it takes 10-15 minutes. This seems to improve on that by:

split out lintrunner-mypy job
copy pytorch/pytorch's changed files tracking
don't run lintrunner init unnecessarily

[ghstack-poisoned]

swolchok · 2025-09-11T23:17:15Z

Stack from ghstack (oldest at bottom):

-> Improve lintrunner job runtime #14240

pytorch-bot · 2025-09-11T23:17:18Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14240

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 Cancelled Jobs, 3 Pending, 5 Unrelated Failures

As of commit d93113d with merge base e6b9111 ():

CANCELLED JOBS - The following jobs were cancelled. Please retry:

Build Presets / linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build (gh)
pull / test-openvino-linux / linux-job (gh)
##[error]The operation was canceled.

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

pull / test-binary-size-linux-gcc / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
pull / test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)
pull / test-selective-build-linux / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / windows / windows-job (gh) (trunk failure)
devtools/inspector/tests/inspector_test.py::TestInspector::test_disable_debug_handle_validation_with_symbolic_shapes
pull / unittest-editable / windows / windows-job (gh) (trunk failure)
devtools/inspector/tests/inspector_test.py::TestInspector::test_disable_debug_handle_validation_with_symbolic_shapes

This comment was automatically generated by Dr. CI and updates every 15 minutes.

lintrunner is required to land, but it takes 10-15 minutes. Can we cut that down if we split out MYPY? ghstack-source-id: f89f9ee ghstack-comment-id: 3282934023 Pull-Request: #14240

.github/workflows/lint.yml

digantdesai · 2025-09-11T23:30:59Z

.github/workflows/lint.yml

+
+        RC=0
+        # Run lintrunner on all files
+        if ! lintrunner --force-color --all-files --skip MYPY --tee-json=lint.json 2> /dev/null; then


I was thinking what if instead of --all-files can we run this on "changed" files in this PR as a required, not sure if that helps with MYPY performance or not..

the problem isn't MYPY performance, it's the time to build and install executorch. check out some recent lintrunner runs on HUD: https://hud.pytorch.org/hud/pytorch/executorch/main/1?per_page=50&name_filter=lintrunner

instead of --all-files can we run this on "changed" files in this PR

good idea, PyTorch's lint.yml does something like this, but it's out of scope for this PR

on second thought, this is a good idea and I'm open to just increasing scope of this PR so that we can be confident that we actually get a speed improvement.

digantdesai

Stamping to unblock you from landing once ready

[ghstack-poisoned]

lintrunner is required to land, but it takes 10-15 minutes. Can we cut that down if we split out MYPY? ghstack-source-id: fe28042 ghstack-comment-id: 3282934023 Pull-Request: #14240

[ghstack-poisoned]

lintrunner is required to land, but it takes 10-15 minutes. Can we cut that down if we split out MYPY? ghstack-source-id: 00a3261 ghstack-comment-id: 3282934023 Pull-Request: #14240

swolchok · 2025-09-12T16:58:02Z

ugh, this only knocks 3 minutes off the time for the lintrunner job. I think we're better off just not running lintrunner init, I'll try that. Also note that these jobs are mysteriously doing nothing for 5 minutes -- there's a huge gap in the log timestamps.

[ghstack-poisoned]

lintrunner is required to land, but it takes 10-15 minutes. Can we cut that down if we split out MYPY? ghstack-source-id: 7acad79 ghstack-comment-id: 3282934023 Pull-Request: #14240

[ghstack-poisoned]

lintrunner is required to land, but it takes 10-15 minutes. Can we cut that down if we split out MYPY? ghstack-source-id: 3de552d ghstack-comment-id: 3282934023 Pull-Request: #14240

swolchok · 2025-09-12T18:02:19Z

@digantdesai can you please rereview? this has had several major changes

larryliu0820 · 2025-09-12T19:52:45Z

.github/workflows/_get-changed-files.yml

+            PR_NUMBER="${{ github.event.number }}"
+
+            # Use gh CLI to get changed files in the PR with explicit repo
+            CHANGED_FILES=$(gh api repos/${{ github.repository }}/pulls/$PR_NUMBER/files --paginate --jq '.[] | select(.status != "removed") | .filename' | tr '\n' ' ' | sed 's/ $//')


Will —paginate give all the files at once or gives a page and ask you to go to next page?

beats me. this is an exact copy from pytorch/pytorch

.github/workflows/lint.yml

lintrunner is required to land, but it takes 10-15 minutes. This seems to improve on that by: - split out lintrunner-mypy job - copy pytorch/pytorch's changed files tracking - don't run `lintrunner init` unnecessarily

Update

6cb1b8e

[ghstack-poisoned]

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 11, 2025

swolchok requested review from huydhn and mergennachin September 11, 2025 23:17

digantdesai reviewed Sep 11, 2025

View reviewed changes

.github/workflows/lint.yml Show resolved Hide resolved

digantdesai reviewed Sep 11, 2025

View reviewed changes

digantdesai approved these changes Sep 11, 2025

View reviewed changes

swolchok added the release notes: none Do not include this in the release notes label Sep 11, 2025

Update

cd18132

[ghstack-poisoned]

don't waste time doing another lintrunner init

aee50cc

[ghstack-poisoned]

port get-changed-files from pytorch/pytorch

e1a3133

[ghstack-poisoned]

syntax is hard

d93113d

[ghstack-poisoned]

swolchok requested a review from digantdesai September 12, 2025 18:02

swolchok changed the title ~~Create lintrunner-mypy so main lintrunner job doesn't have to build~~ Improve lintrunner job runtime Sep 12, 2025

larryliu0820 reviewed Sep 12, 2025

View reviewed changes

larryliu0820 approved these changes Sep 12, 2025

View reviewed changes

swolchok merged commit eb1099e into main Sep 12, 2025
116 of 125 checks passed

swolchok deleted the gh/swolchok/576/head branch September 12, 2025 20:07

digantdesai reviewed Sep 12, 2025

View reviewed changes

.github/workflows/lint.yml Show resolved Hide resolved

.github/workflows/lint.yml Show resolved Hide resolved

Improve lintrunner job runtime #14240

Improve lintrunner job runtime #14240

Uh oh!

Conversation

swolchok commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swolchok commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14240

❌ 2 Cancelled Jobs, 3 Pending, 5 Unrelated Failures

Uh oh!

Uh oh!

digantdesai Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

swolchok Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

swolchok Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

swolchok Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

swolchok commented Sep 12, 2025

Uh oh!

swolchok commented Sep 12, 2025

Uh oh!

larryliu0820 Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

swolchok Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

swolchok commented Sep 11, 2025 •

edited

Loading

swolchok commented Sep 11, 2025 •

edited

Loading

pytorch-bot bot commented Sep 11, 2025 •

edited

Loading