ci: Introduced vetting test on PRs and corrected package version to match "current state of the package" #3549
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
michalChrobot
requested review from
a team,
EmandM and
NoelStephensUnity
as code owners
|
We can land this after 2.5.0 release so we are "up to date" |
|
FYI I included an improvement of release.py to address MTT-12840 and make the package "release ready" with running this script. I'm waiting for 2.5.0 to be released first |
michalChrobot
added a commit
that referenced
this pull request
Jul 31, 2025
…ersion to match "current state of the package" (#3550) This PR is a backport of #3549 and introduces vetting test that should be running on PR to confirm that the package will be in "releasable state". Things to note are ## Bumping package version After N4E investigation in I noticed that vetting tests were not working properly because after each release we should bump package version to match "current state of the package" which means bumping it by a patch. That way the current package version is being compared against last released. Otherwise this test won't give us any valid result because we will be comparing 2 already released packages. Because of that I bumped NGO internal version to 2.5.0 and in that way we will be comparing this "current" version against last released 2.4.3 ## What this test will do This test will mostly evaluate API compatibility. What we want to achieve is to not release any new minor version if not needed/detect all new APIs earlier. Notice that this test will fail if any new API is introduced (because for new API we need to release new minor and not patch) and in such scenario we should either - Bump package version to new minor, indicating that we will release new minor version - Make the API internal/private This will ensure that we know the current state of the package version. Notice that in theory we could also use wrench/api-validation test but this would require manually modifying this file after every wrench update (to add release.py execution) so easier approach will be to create a dedicated test and leave wrench one for actual release validation ## release.py script This python script is basically copied from N4E repo and updates CHANGELOG and any other places that need to be formatted for the release. For now it's only used for this test (to avoid it throwing an error about CHANGELOG being improperly formatted) but in the future we can use it to automate releasing ## Backport #3549
|
Ok, since we are not 100% sure when we will land 2.5.0 I decided to just merge this PR so I can proceed with automations. I corrected the package version to |
michalChrobot
added a commit
that referenced
this pull request
Sep 12, 2025
## Purpose of this PR This PR automates the releases of NGO package (which can be extended for other packages since it's quite modular). The job will operate on the branch from which it will be triggered and by default it will be on the `develop` branch (job trigger). For more background note that Netcode team sprint last 4 weeks, starting and ending on Wednesday and our goal is to kickstart the release process after last full week of the sprint and preferably release until next Wednesday (end of the sprint). So the goal would be to run everything on Saturday (after the week concluded) and have everything ready from Monday morning to Playtest/investigate failures. As a separate part of this automation, as per **[THIS PR](#3557 we are able to trigger builds of our samples so those will also be triggered on the weekend to be ready for playtesting on Monday. Now with this in mind the process will work in the following way: 1. It runs on weekly schedule but we are interesting on executing it every 4th Saturday (after last full week of the Netcode team sprint. Our sprint last 4 weeks). Because of that the first action that this job will do is to run a check to see if the following requirements are satisfied: - Is this the 4th Saturday since we last released? - Is NGO CHANGELOG not empty? (no point of making the release is there is nothing to release) - Is the release branch not yet created? Before looking at the next steps, please note that as per **[THIS PR](#3549 the package version of Nshould always correspond to the current state of the package. 2. If the above is satisfied the next step will be to create the release branch (based on the known package version). The already existing **release.py** script will be executed there in order to regenerate recipes, cleanup changelog and generally prepare our package for releasing (see result **[HERE](c9f3f204fc4058df8781dd577b187a117216b544)**) 3. The next step will be to trigger `all_promotion_related_jobs_promotiontrigger` job. In that way when we will set-up packageworks release stream on Monday morning we will already have the results from this job and we can address any potential failures. 4. As a last step the job will create a commit to the local branch (normally `develop` but for the sake of playtesting I used a test branch). The goal with changelog here is we want to avoid changelog divergence if release will take longer then expected. The script will clean the empty sections, assign correct release version and date and then add back the [Unreleased] section template for the purpose of next entries. The goal with package version is that we want to indicate that now the package reflects further state (next patch). After this the job will commit this to the branch. I created Netcode-Team-Bot with appropriate tokens and marked committer to point to the bot (when for example making a direct commit to `develop` with changelog and package version update) <img width="1782" height="292" alt="image" src="https://github.com/user-attachments/assets/81e76d5a-b8f3-48f2-985f-40126e6511b8" /> The bot will make commits like <img width="1804" height="164" alt="image" src="https://github.com/user-attachments/assets/728eb89c-2c59-4b81-8a7e-7f858470b979" /> ## Jira ticket https://jira.unity3d.com/browse/MTT-12841 ## Documentation After this gets merged I will add approperiate description of Netcode release process in https://docs.google.com/document/d/16g9B5jxXeV0zG44Fhax-6Md_gtLoqGVthKQ7LkDvdqM/edit?tab=t.0#heading=h.t1guku6agyev ## Testing & QA Since this automation runs on the same branch on which it's triggered (and just because of job trigger it will be `develop-2.0.0` branch) I tested it locally (each script) and entire automation was tested on `testing-automated-netcode-releases` branch, the effects of which you can see as follows (note that instead of `develop-2.0.0` I just used local branch): 1. I tested for the normal release schedule date (result **[HERE](https://unity-ci.cds.internal.unity3d.com/job/56127622)**) 2. I tested with empty changelog (result **[HERE](https://unity-ci.cds.internal.unity3d.com/job/56123485)**) 3. The job itself was green (**[LINK](https://unity-ci.cds.internal.unity3d.com/job/56127745)**). 4. `release/2.5.1` branch was created with proper initial setup (**[LINK](https://github.com/Unity-Technologies/com.unity.netcode.gameobjects/tree/release/2.5.1)**), see latest commits there 5. Changelog and package versions were updated on `develop-2.0.0` branch (replaced for testing with `testing-automated-netcode-releases`). Notice that this simulates what will happen on develop branch after this script runs. (**[LINK](https://github.com/Unity-Technologies/com.unity.netcode.gameobjects/tree/testing-automated-netcode-releases-2.x)**) 6. Wrench jobs were correctly triggered on the created release branch (**[LINK](https://unity-ci.cds.internal.unity3d.com/project/1201/branch/release%2F2.5.1/jobDefinition/.yamato%2Fwrench%2Fpromotion-jobs.yml%23publish_dry_run_netcode_gameobjects/recent-jobs?nav=none)**) 7. Builds were triggered from release branch (**[example build LINK](https://unity-ci.cds.internal.unity3d.com/project/1201/branch/release%2F2.5.1/jobDefinition/.yamato%2Fproject-builders%2Fproject-builders.yml%23build_BossRoom_project/recent-jobs?nav=jobDefinitions)**) ## Things to note - Updating package version to next patch won't be immediately valid (vetting test will still compare it to latest released package version) but it serves the purpose of automating those steps and not needing to remember about it after package is released - **[email protected]** was created to assign NETCODE_YAMATO_API_KEY to it. Additionally I created tokens for both GitHubs and when making commit I'm just pointing to this bot as committer and author. ## Backports It's backported to #3603
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces vetting test that should be running on PR to confirm that the package will be in "releasable state". Things to note are
Bumping package version
After N4E investigation in I noticed that vetting tests were not working properly because after each release we should bump package version to match "current state of the package" which means bumping it by a patch. That way the current package version is being compared against last released. Otherwise this test won't give us any valid result because we will be comparing 2 already released packages.
Because of that I bumped NGO internal version to 2.5.0 and in that way we will be comparing this "current" version against last released 2.4.3
What this test will do
This test will mostly evaluate API compatibility. What we want to achieve is to not release any new minor version if not needed/detect all new APIs earlier. Notice that this test will fail if any new API is introduced (because for new API we need to release new minor and not patch) and in such scenario we should either
This will ensure that we know the current state of the package version. Notice that in theory we could also use wrench/api-validation test but this would require manually modifying this file after every wrench update (to add release.py execution) so easier approach will be to create a dedicated test and leave wrench one for actual release validation
release.py script
This python script is basically copied from N4E repo and updates CHANGELOG and any other places that need to be formatted for the release. For now it's only used for this test (to avoid it throwing an error about CHANGELOG being improperly formatted) but in the future we can use it to automate releasing
Backport
#3550