Adding first version for which clang-tidy checks and their options are available

[0hDEADBEAF]

As discussed on discourse, I have been working on adding the first version for which each clang-tidy checks and their options are available in the documentation.

I have made a Python script (not included in that PR) to track down the available checks/options for each LLVM releases (using the GitHub API to accurately gather/download the releases and then building them from sources).

The draft includes a lot of changes but since those changes were automatically generated by the script, they can easily be tuned if you think they should.

Here is a small summary of the modifications I have made:

• I added, for each clang-tidy check, the first version for which it was available (see clang-tools-extra/docs/clang-tidy/checks/list.rst file).
• I added, for each clang-tidy check option, the first version for which it was available (see every files in clang-tools-extra/docs/clang-tidy/checks subfolders).
• I added, for practical reasons (most people using the online documentation are using older versions of clang-tidy), the first version from which clang-tidy checks were removed, clearly indicating (strike-through) that they are no longer available. I had to modify clang-tools-extra/docs/_static/clang-tools-extra-styles.css file and add the following code to clang-tools-extra/docs/clang-tidy/checks/list.rst file to allow me to use strike-through:

.. role:: strike
   :class: strike

As I was working on this PR, few questions came to my mind:

1. Should I include the Python script I have made to generate those changes in this PR? Or maybe just share it with Gist? This would allow you to verify it to make sure the information I added is correct. I am not sure this script will ever be useful in the future as long as we make sure every newly added/removed check/option includes those information so maybe there is no need to add it in the code base.
2. I have also added information about removal of checks/options, which was not planned at first but since most users are not using the clang-tidy version corresponding to the online documentation, I believe this could be a nice add. Are you ok with that? Regarding the way to display it, I could make a separate category that would only contain the removed checks if you think this is better this way (for now, they are listed among the available checks, with strike-through and displaying the version on which it was removed).
3. In order to track down these pieces of information, I have built every clang-tidy/clang-tools-extra releases since 7.1.0 from sources. From what I have seen, people can contribute to those public releases and share their builds to make them available for everyone. I was wondering if there was a specific procedure to follow to submit my builds. If so, could you please tell me what steps to take?

I am not sure whether I should mention people (and who) for this draft PR but since you have been following this project on that Discourse discussion, I will invoke you @vbvictor! :p

[vbvictor]

Hi, right now I have limited ability to process everything, so I wouldn't answer to all questions, sorry. But I have some general comments on how we should proceed.

Let's go with small, separate PRs to focus on one thing at a time:

1. Upstream this script for review. In case someone notices a bug, we would be able to adjust the script and re-run on docs.
2. Make changes to add_new_check.py so that people generate new checks with proper versions
3. Run script on per-module basis to have fewer changes to review. Note that I don't think that we need "minor" and "patch" versions, we should go with only major version, like in clangd.

[0hDEADBEAF]

Hello vbvictor!
Thank you for your repsonse, it is now more clear how should I proceeed. Answering your points:

1. I am sorry, I am not quite sure to understand what you mean by

Upstream this script for review.

Does that mean I should make a separate PR with only this script for people to review it? Or upload it somewhere (Gist?) and share it for review (if so, which platform should I share it on? Discourse? Discord?).
Regardless of that, I still have some modifications/cleanups to do regarding it and I also need to make sure it follows LLVM guidelines.

2. To make sure I understand properly what you mean: once the script is reviewed and approved, I should merge it with add_new_check.py file so the versions are then automatically generated for the docuementation?

3. Alright, sounds good to me! If we omit minor and patch versions, should I consider that LLVM v15 is referring to the latest v15 release (15.0.7) or the first one (15.0.0)? In case it ever matters and actually changes what checks/options are available. I feel like using the latest version (in this case, 15.0.7) would make the most sense.

[vbvictor]

Does that mean I should make a separate PR with only this script for people to review it? Or upload it somewhere (Gist?) and share it for review (if so, which platform should I share it on? Discourse? Discord?).
Regardless of that, I still have some modifications/cleanups to do regarding it and I also need to make sure it follows LLVM guidelines.

I think we can put it as a PR in LLVM, it's the most convenient place to look at code. The benefits of review are:

• We (clang-tidy team) can verify that it works correctly and have a dedicated place to share opinions.
• If we find out a bug (maybe by user bug report), we can fix the script ourselves and rerun it over all documents to fix everything. This is particularly important in case you will not be available. If we don't store it here, your Gist can be deleted because you got hacked/deleted etc... I think we as a project can't depend on 3rd party software that has a single point of failure.

[vbvictor]

I think we have a pretty low quality bar for scripts (not professional python devs) so it should be quite easy to follow LLVM style guide. You can use LLM to make your script "look like" other scripts in clang-tidy directory.

[vbvictor]

To make sure I understand properly what you mean: once the script is reviewed and approved, I should merge it with add_new_check.py file so the versions are then automatically generated for the docuementation?

Yes, we should patch add_new_check.py so that it would automatically set current LLVM version in proper places. When making new checks, developers shouldn't think about "what version should I put?".

How we patch add_new_check.py script depends on complexity: we can just copy-paste needed functions in add_new_check.py from your script

[vbvictor]

Alright, sounds good to me! If we omit minor and patch versions, should I consider that LLVM v15 is referring to the latest v15 release (15.0.7) or the first one (15.0.0)? In case it ever matters and actually changes what checks/options are available. I feel like using the latest version (in this case, 15.0.7) would make the most sense.

Yes, let's go for the latest version (15.0.7).
We should also decide from what version we should start. AFAIK, clangd starts documenting from clangd-11. Do you have a version in your mind? I looked at clang versioning and probably 4 is the right place to start from because LLVM started to increase MAJOR version every new release opposed to MINOR version increase for each new 3.x.x release.

[0hDEADBEAF]

I think we can put it as a PR in LLVM, it's the most convenient place to look at code. The benefits of review are:

• We (clang-tidy team) can verify that it works correctly and have a dedicated place to share opinions.
• If we find out a bug (maybe by user bug report), we can fix the script ourselves and rerun it over all documents to fix everything. This is particularly important in case you will not be available. If we don't store it here, your Gist can be deleted because you got hacked/deleted etc... I think we as a project can't depend on 3rd party software that has a single point of failure.

Ok it makes sense indeed.

[0hDEADBEAF]

Yes, let's go for the latest version (15.0.7). We should also decide from what version we should start. AFAIK, clangd starts documenting from clangd-11. Do you have a version in your mind? I looked at clang versioning and probably 4 is the right place to start from because LLVM started to increase MAJOR version every new release opposed to MINOR version increase for each new 3.x.x release.

Since my script is based on releases listed on this GitHub repository, the first version I got information from is the v7. However, I can modify it to get information from the version 4, it should not be too hard to fetch it.