Document PGO policy regarding explicit user optimization hint behavior

[AaronBallman]

We document PGO at https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization but we do not document the behavior of PGO in the presence of user-provided optimization hints. It would be good to document how PGO handles conflicts between the hints the user explicitly wrote in their code and the profile they're optimizing against. For example, how do the following (non-exhaustive) hints behave: [[likely]] and other optimization attributes, register or inline use, __builtin_expect, etc.

This will help to set user expectations -- some users expect explicit optimization hints to be ignored by PGO while other users assume the hints are still honored (at least sometimes). If hints are honored, they should be clearly documented as doing so if that behavior is intentional.

[zamazan4ik]

@AaronBallman before we have the actual documentation in Clang, could you please write at least here - what is the current Clang/LLVM behavior in this case? It would be really interesting to know at least as a part of the GitHub issue.

If the current behavior is unknown, could we ping someone here to clarify the question?

[AaronBallman]

@AaronBallman before we have the actual documentation in Clang, could you please write at least here - what is the current Clang/LLVM behavior in this case? It would be really interesting to know at least as a part of the GitHub issue.

If the current behavior is unknown, could we ping someone here to clarify the question?

CC @bogner @dnovillo as LLVM code owners for profile data.

[travisdowns]

Would love to see this doc. E.g., I have a few questions:

• If PGO is enabled, will likely/unlikely annotations affect the calculated branch probability at all for branches which are visited in the training run?
• If PGO is enabled, how are likely/unlikely annotations used for branches which are not visited during the training run? E.g., are they ignored (as the documentation suggests) or are they used in a similar manner (for those branches) to a non-PGO compilation, or something else?

[zamazan4ik]

If PGO is enabled, will likely/unlikely annotations affect the calculated branch probability at all for branches which are visited in the training run?

I've made such tests before. For Clang, the PGO profile has higher priority than users annotations. However, be careful - it's just a conclusion from a simple tests without any proof from the actual implementation. I confirmed such results for both Clang and Rustc.

If PGO is enabled, how are likely/unlikely annotations used for branches which are not visited during the training run? E.g., are they ignored (as the documentation suggests) or are they used in a similar manner (for those branches) to a non-PGO compilation, or something else?

Right now, not visited branches are considered as cold - see this one. However, there is a trick to avoid this.

How it works with likely/unlikely in this case - good question! I need to test it. My guess is that the attributes are ignored and branches are still considered as cold.

I hope it answers your questions! However, it definitely would be nice to see such details in the documentation. As for now, I am just trying to collect all such tricky details in awesome-pgo (like this entry about Clang).