[CI/Build] Add bc-linter to vLLM CI #21234

zhewenl · 2025-07-19T23:12:11Z

Summary:
Add bc-linter to vLLM CI, in this PR we are piloting adding this only to vllm/v1/core/sched/output.py
Config is defined in .bc-linter.yml, where https://github.com/pytorch/test-infra/blob/main/tools/stronghold/docs/bc_linter_config.md has more examples

Class support, config support added in pytorch/test-infra#6953, pytorch/test-infra#7016

Purpose

bc-linter(https://github.com/pytorch/test-infra/wiki/BC-Linter) is tool in pytorch test infra to test backward compatibility on public pytorch functions used externally. this PR enables the check in vLLM where we can use .bc-linter.yml with @bc_linter_include / @bc_linter_excludeannotations to protect bc on functions

Test Plan

Test Plan:
CI

Test Result

https://github.com/vllm-project/vllm/pull/23855/files#diff-cafd89ce8a698a56acb24ada62831cbc7a980782f78a52d1742ba238031f296c

gemini-code-assist · 2025-07-19T23:12:17Z

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

github-actions · 2025-07-19T23:12:18Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

yeqcharlotte · 2025-07-20T04:00:38Z

.github/workflows/bc-lint.yml

@@ -0,0 +1,23 @@
+name: BC Lint
+
+on:


possible to only turn on for specific folders only?

i think remind folks of BC breaking changes in core/ and attention/ are generally helpful. while it might feel a bit spammy in other folders.

yeqcharlotte · 2025-07-21T07:14:27Z

.github/workflows/bc-lint.yml

+      - vllm/attetion/**
+      - vllm/core/**


v0 is not needed

yeqcharlotte

LGTM! Can we also update the lint message so folks know how to suppress the check?

cc: @huydhn @seemethere @simon-mo @WoosukKwon to also take a look!

houseroad · 2025-07-21T07:21:21Z

.github/workflows/bc-lint.yml

+    # branches-ignore:
+    #   - <branches to ignore bc-linter>
+    paths:
+      - vllm/attetion/**


Does this mean we have all the files covered here to ensure the BC guard?

Yep, it will check all the files under this path

houseroad

It's not clear to me, how the BC checking works? Do we need to tag some function to be BC guaranteed?

zhewenl · 2025-07-21T20:36:27Z

Discussed offline, will check if we can use some annotations to restrict linter coverage for a selection of tests

It's not clear to me, how the BC checking works? Do we need to tag some function to be BC guaranteed?

@houseroad here is documentation of bc-linter: https://github.com/pytorch/test-infra/wiki/BC-Linter

The linter is designed to identify and report issues such as function deletion, parameter removal, renaming, or reordering, changes in parameter requiredness, and removal of support for variadic parameters.

like some warnings in https://github.com/vllm-project/vllm/pull/21235/files#diff-bafd6bd048c5ca7bad726e6f509fe4409f4033342e2b5b0c3016156e2f708b12

huydhn · 2025-07-23T00:00:16Z

.github/workflows/bc-lint.yml

+          repo: ${{ github.event.pull_request.head.repo.full_name }}
+          base_sha: ${{ github.event.pull_request.base.sha }}
+          head_sha: ${{ github.event.pull_request.head.sha }}
+          suppression: ${{ contains(github.event.pull_request.labels.*.name, 'suppress-bc-linter') }}


Just a note that using labels like suppress-bc-linter is a common self-serve mechanism in PyTorch PR, but we would need a maintainer to do it here, like adding the ready label, which is ok

huydhn · 2025-07-23T00:01:58Z

.github/workflows/bc-lint.yml

+      - vllm/v1/core/**
+
+jobs:
+  bc_lint:


You want this line https://github.com/pytorch/pytorch/blob/main/.github/workflows/lint-bc.yml#L19C5-L19C45 to run this job only on PR submitted to vllm

Suggested change

bc_lint:

bc_lint:

if: github.repository_owner == 'vllm-project'

huydhn · 2025-07-23T00:03:24Z

.github/workflows/bc-lint.yml

+          base_sha: ${{ github.event.pull_request.base.sha }}
+          head_sha: ${{ github.event.pull_request.head.sha }}
+          suppression: ${{ contains(github.event.pull_request.labels.*.name, 'suppress-bc-linter') }}
+          docs_link: 'https://github.com/pytorch/test-infra/wiki/BC-Linter'


Also the concurrency rule is important https://github.com/pytorch/pytorch/blob/main/.github/workflows/lint-bc.yml#L31-L33 to avoid multiple bc linter jobs running at the same time on the same PR

Suggested change

docs_link: 'https://github.com/pytorch/test-infra/wiki/BC-Linter'

docs_link: 'https://github.com/pytorch/test-infra/wiki/BC-Linter'

concurrency:

group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}

cancel-in-progress: true

huydhn · 2025-07-23T00:16:18Z

Have you been able to run this https://github.com/pytorch/test-infra/blob/main/.github/actions/bc-lint/action.yml#L62-L68 locally?

At a high level, the BC linter uses https://docs.python.org/3/library/ast.html to parse the python changes in the PR and check for those changes https://github.com/pytorch/test-infra/blob/main/tools/stronghold/src/api/violations.py. It has some PyTorch-specific logic that might not be applicable to vLLM https://github.com/pytorch/test-infra/blob/main/tools/stronghold/src/api/compatibility.py#L20-L43, but we could easily update that if needed, for example, it ignores anything start with _NAME because it's a PyTorch convention to have non-standard API starting with underscore (cc @houseroad fyi)

Signed-off-by: zhewenli <[email protected]>

mergify bot added the ci/build label Jul 19, 2025

zhewenl self-assigned this Jul 19, 2025

zhewenl requested review from yeqcharlotte and houseroad July 19, 2025 23:16

add bc linter to github workflow

6a602e1

zhewenl force-pushed the add-bc-linter branch from 0b3082d to 6a602e1 Compare July 19, 2025 23:19

update yml

eb568db

yeqcharlotte reviewed Jul 20, 2025

View reviewed changes

zhewenl added 2 commits July 20, 2025 23:09

update path

9e37544

update v1 path

68cca75

yeqcharlotte reviewed Jul 21, 2025

View reviewed changes

yeqcharlotte approved these changes Jul 21, 2025

View reviewed changes

houseroad reviewed Jul 21, 2025

View reviewed changes

houseroad requested changes Jul 21, 2025

View reviewed changes

huydhn reviewed Jul 23, 2025

View reviewed changes

zhewenl added 2 commits August 28, 2025 10:48

Merge branch 'vllm-project:main' into add-bc-linter

564cfb2

update bc linter

db79acd

zhewenl requested review from WoosukKwon, robertgshaw2-redhat, njhill, ywang96, comaniac and alexm-redhat as code owners August 28, 2025 18:41

add bc yml

6ec6b32

mergify bot added the v1 label Aug 28, 2025

zhewenl added 4 commits August 28, 2025 11:57

format

b719ba2

Signed-off-by: zhewenli <[email protected]>

update bc linter

8a24857

Signed-off-by: zhewenli <[email protected]>

update bc linter

91136a5

Signed-off-by: zhewenli <[email protected]>

update bc linter

70ee9a0

Signed-off-by: zhewenli <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

[CI/Build] Add bc-linter to vLLM CI #21234

[CI/Build] Add bc-linter to vLLM CI #21234

zhewenl commented Jul 19, 2025 •

edited by github-actions bot

Loading

Uh oh!

gemini-code-assist bot commented Jul 19, 2025

Uh oh!

github-actions bot commented Jul 19, 2025

Uh oh!

yeqcharlotte Jul 20, 2025

Uh oh!

yeqcharlotte Jul 21, 2025 •

edited

Loading

Uh oh!

yeqcharlotte left a comment •

edited

Loading

Uh oh!

houseroad Jul 21, 2025

Uh oh!

zhewenl Jul 21, 2025

Uh oh!

houseroad left a comment

Uh oh!

zhewenl commented Jul 21, 2025 •

edited

Loading

Uh oh!

huydhn Jul 23, 2025

Uh oh!

huydhn Jul 23, 2025 •

edited

Loading

Uh oh!

huydhn Jul 23, 2025

Uh oh!

huydhn commented Jul 23, 2025 •

edited

Loading

Uh oh!

Uh oh!

	bc_lint:
	bc_lint:
	if: github.repository_owner == 'vllm-project'

-          docs_link: 'https://github.com/pytorch/test-infra/wiki/BC-Linter'
+          docs_link: 'https://github.com/pytorch/test-infra/wiki/BC-Linter'
+ concurrency:
+  group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
+  cancel-in-progress: true

Uh oh!

[CI/Build] Add bc-linter to vLLM CI #21234

Are you sure you want to change the base?

[CI/Build] Add bc-linter to vLLM CI #21234

Conversation

zhewenl commented Jul 19, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

gemini-code-assist bot commented Jul 19, 2025

Uh oh!

github-actions bot commented Jul 19, 2025

Uh oh!

yeqcharlotte Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

yeqcharlotte Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

yeqcharlotte left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

houseroad Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

zhewenl Jul 21, 2025

Choose a reason for hiding this comment

Uh oh!

houseroad left a comment

Choose a reason for hiding this comment

Uh oh!

zhewenl commented Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Discussed offline, will check if we can use some annotations to restrict linter coverage for a selection of tests

Uh oh!

huydhn Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

huydhn Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

huydhn Jul 23, 2025

Choose a reason for hiding this comment

Uh oh!

huydhn commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

zhewenl commented Jul 19, 2025 •

edited by github-actions bot

Loading

yeqcharlotte Jul 21, 2025 •

edited

Loading

yeqcharlotte left a comment •

edited

Loading

zhewenl commented Jul 21, 2025 •

edited

Loading

huydhn Jul 23, 2025 •

edited

Loading

huydhn commented Jul 23, 2025 •

edited

Loading