Reconsider using t1 rather than t2 for landing pad label register

[kito-cheng]

The reason use t2 rather than t1 is it could prevent disturb PLT too much - so t2 is better choice than t1 - however I found we need to defined new PLT stub, also tweak little bit for _dl_runtime_resolve[2] - so it already need to adjust few things.

PLT already need to define new stubs sequence, and _dl_runtime_resolve also need to update too, so the reason using t2 is not really so strong, BUT the price is we need to change the expansion of tail, which is make assembler inconsistently here.

Also we found that may cause more problem than our expect on using t2 (riscv-non-isa/riscv-elf-psabi-doc#425 (comment)) at psABI PR.

So again, I think we should reconsider using t1 rather than t2 for landing pad label register - also I believe it's not big disturb to the ISA spec itself.

[1] #125 (comment)
[2] sifive/riscv-glibc@4d237e2
[3] riscv-non-isa/riscv-asm-manual#93

[kito-cheng]

cc @sorear @jrtc27 @rui314 @aswaterman @ved-rivos

[ved-rivos]

If we used t1/x6 as label register then we would break ABI compatibility since _dl_runtime_resolve expects to receive the relocation offset in t1/x6 and we would have had to change that if x6 was used as label register since label register cannot be clobbered. The tail expansion as I understand is localized to the binary - is that right? The choice of t2 allowed for a program compiled with Zicfilp to be loaded on a system that does not support Zicfilp and so does not need an updated dynamic loader. So while both changes - changing tail expansion or using t1 as landing pad label register - have an ABI impact, the impact of using t2 was more localized to tail expansion and change to PLT is also localized to that binary. Is that not right?

[kito-cheng]

@ved-rivos, thanks for the explain, I think I got the point, the compatibility for running new binary with legacy platform, which is the benefit we can get from using t2, but it will not able to compatible IF we use t1.

Currently we have some problem when using t2, but I think they are mostly local issue rather than the cross ABI boundary issue, let me list all concern we have before:

• tail: use t1 before, must change to t2 when zicfilp enabled.
  • As Ved mention it's only effect locally, the biggest controversy is that may violate the convention before.
• LLVM using t2 to pass argument for the internal calling convention.
  • That's the compiler internal only, so I think that's fine to disturb that because it's internal calling convention, not need to guarantee must stable.
• GCC/LLVM using t2 as static chain for trampoline function.
  • This has discussed within psABI before, to see if it need write down into spec, but it's kinda used in compiler internal.

I was confused by "ABI compatible", the PLT stub must update, so I thought we already got some ABI incompatible, but it's not really main point.

Anyway I saw we pay the price of using t2 for tail...and we can get the compatibility for running new binary with CFI on the old platform.

Thanks Ved

[jrtc27]

@ved-rivos, thanks for the explain, I think I got the point, the compatibility for running new binary with legacy platform, which is the benefit we can get from using t2, but it will not able to compatible IF we use t1.

Currently we have some problem when using t2, but I think they are mostly local issue rather than the cross ABI boundary issue, let me list all concern we have before:

• tail: use t1 before, must change to t2 when zicfilp enabled.

  • As Ved mention it's only effect locally, the biggest controversy is that may violate the convention before.

• LLVM using t2 to pass argument for the internal calling convention.

  • That's the compiler internal only, so I think that's fine to disturb that because it's internal calling convention, not need to guarantee must stable.

• GCC/LLVM using t2 as static chain for trampoline function.

  • This has discussed within psABI before, to see if it need write down into spec, but it's kinda used in compiler internal.

I was confused by "ABI compatible", the PLT stub must update, so I thought we already got some ABI incompatible, but it's not really main point.

Anyway I saw we pay the price of using t2 for tail...and we can get the compatibility for running new binary with CFI on the old platform.

Thanks Ved

Isn't that last point a serious problem that would make Zicfilp code ABI-incompatible with existing GCC-compiled nested function code?

[kito-cheng]

Isn't that last point a serious problem that would make Zicfilp code ABI-incompatible with existing GCC-compiled nested function code?

Nested function isn't cross the ABI boundary which only impact within module, which won't cause cross module problem, I mean that's is something like fastcc issue?

[deepak0414]

Last I remember GCC implements nested function by making stack executable (I remember that on x86) and placing a trampoline on stack. I am not sure if that has changed.

[kito-cheng]

Last I remember GCC implements nested function by making stack executable (I remember that on x86) and placing a trampoline on stack. I am not sure if that has changed.

Yeah, that's still true for RISC-V

[sorear]

Strictly speaking _dl_runtime_resolve is also a per-binary consideration since it isn't accessed by name; a dynamic loader can have more than one version and set the resolver pointer in the GOT appropriately for each binary.

tail is kinda irrelevant for compatibility questions since it's just assembler syntax.

GCC only uses executable stack for nested functions if the address of the nested function is taken. Otherwise, GCC will generate an ordinary symbol for the nested function, call instructions, and R_RISCV_CALL relocations to call it after setting t2. The psABI states that R_RISCV_CALL has the same behavior as R_RISCV_CALL_PLT and might go through a PLT stub, which means that GCC is effectively assuming that t2 survives PLT stubs.

Remember there is an ABI boundary between the compiler and the linker, not just the linker and the dynamic loader. I don't think it's accurate to describe GCC nested functions and LLVM fastcc as "compiler internal"; the symbols and relocations are exposed to the linker, and need to respect linker semantics.

[kito-cheng]

Strictly speaking _dl_runtime_resolve is also a per-binary consideration since it isn't accessed by name; a dynamic loader can have more than one version and set the resolver pointer in the GOT appropriately for each binary.

New libc with old binary can work with the way you mention I think, but the point is we can use existing/old libc version with the binary compile with landing pad.

GCC only uses executable stack for nested functions if the address of the nested function is taken. Otherwise, GCC will generate an ordinary symbol for the nested function, call instructions, and R_RISCV_CALL relocations to call it after setting t2. The psABI states that R_RISCV_CALL has the same behavior as R_RISCV_CALL_PLT and might go through a PLT stub, which means that GCC is effectively assuming that t2 survives PLT stubs.

I would say those function is not standard CC conformance, even we don't give those function with STO_RISCV_VARIANT_CC yet, and those function should also be non-global function, so it's won't cross the ABI boundary IMO.

Remember there is an ABI boundary between the compiler and the linker, not just the linker and the dynamic loader. I don't think it's accurate to describe GCC nested functions and LLVM fastcc as "compiler internal"; the symbols and relocations are exposed to the linker, and need to respect linker semantics.

But I do know what's you mean here...especially something like range extension thunk stuffs, that's kinda psABI debt we need to paid (on psABI TG side) :(