Make tests easier to run #4620
Open
Make tests easier to run #4620
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Author
|
@microsoft-github-policy-service agree company="Indel AG" |
Author
|
I had forgotten to run Format Document again on modified TypeScript files. Fixed in this force push. |
Server Mode was removed in favor of the File API in CMake 3.20, so skip the respective tests when it is not supported. Whether server mode is supported was tracked already, but only with a lower version bound. Availability of server mode is explicitly named in the `-E capabilities` response, so additionally take that into account, effectively adding an upper bound.
Skip configure-time compiler checks when passing a fake sysroot to the compiler, they would fail because it is unable to link an executable that way. I am not sure under what circumstances this has ever worked.
Make the tests that rely on executables in test/fakebin/ check whether they exist, and if not give a helpful error message, instead of failing in unclear ways when they find things in the system $PATH instead. Add a command `yarn pretest-buildfakebin` that documents how to build these executables, which was otherwise only documented in .github/workflows/ci-main*.yml and .vscode/settings.json.
Author
|
Rebased on main to fix merge conflicts: Remove the first commit because equivalent changes have in the meantime been made as part of #4633. No other changes required. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This changes the developer experience (but not the user experience)
The following changes are proposed:
Lift cmake_minimum_required of test cases to 3.5.0(More details in the individual commit messages.)
The purpose of this change
Make it easier for a new developer of CMake Tools to run tests.
Other Notes/Information
I was working on a new feature for CMake Tools and, being new to the project, was having some trouble running the tests. After some investigation, it all came down to two hurdles:
To fix 1., instead of just writing instructions to install CMake 3.18, I decided to make the tests work with the current version 4.2, because the feature I am working on requires CMake 4.3 (not released yet) and I will want to test with that too. The changes required for that turned out to be relatively minor, and are contained in the first
threetwo of the attached commits.To fix 2., I made the tests that without fakebin executables would fail in obscure ways, instead fail with a clear message telling the user how to build the missing executables. This is the last commit.
Together, these changes allow me to successfully run all tests (
yarn smokeTests,unitTests,integrationTests,endToEndTestsSuccessfulBuild,endToEndTestsSingleRoot,endToEndTestsMultiRoot,backendTests) with both CMake 4.2.0 and 3.18.6 on my Ubuntu 20.04 system.Would these changes be welcome in the project? Or would you prefer other ways to mitigate these hurdles?
Since these are not user-visible changes, I did not add an entry to CHANGELOG.md. Should I?