Codify Save/Restore Frame Layout

These layouts need to be compatible between what the compiler expects
when emitting CFI information, and what the library actually implements,
so we write a bit more detail about how these should work to ensure that
compatibility.

Fixes #35

Signed-off-by: Sam Elliott <[email protected]>

Author: lenary

src/toolchain-conventions.adoc

@@ -193,9 +193,12 @@ have the following signatures:
 - `void` `+__riscv_restore_<N>(void)+`
 - `void` `+__riscv_restore_tailcall_<N>+` `(void * tail /* passed in t1 */)` (LLVM/compiler-rt only)
 
-`<N>` is a value between 0 and 12 and corresponds to the number of
-registers between `s0` and `s11` that are saved/restored. The return
-address register `ra` is always included in the registers saved and restored.
+`<N>` is a value between 0 and 12 and corresponds to the number of registers
+between `s0` and `s11` that are saved/restored. When using the `ILP32E` ABI,
+`<N>` can be at most 2, as `s3` to `s12` are temporary registers in this ABI.
+
+The return address register `ra` is always included in the registers saved and
+restored.
 
 The `+__riscv_save_<N>+` functions are called from the prologue, using `t0` as
 the link register to avoid clobbering `ra`. They allocate stack space for the
@@ -216,6 +219,37 @@ As of November 2021 the additional tail-call entry points are only
 implemented in compiler-rt, and calls will only be generated by LLVM
 when the option `-mllvm -save-restore-tailcall` is specified.
 
+=== Save Restore Routine Stack Frame Layouts
+
+While the implementation of the save restore routines are in the library, it is
+the compiler's responsibility to emit the unwind information (CFI) for the
+registers that are saved and restored by these routines, so the compilers and
+the libraries must agree on the stack layouts used by these routines.
+
+As the stack pointer must be correctly aligned at all times, the save restore
+routines are expected to allocate more stack than they require to spill all
+registers in many cases. Additional Callee-saved registers, beyond those
+requested, may be saved and restored by these routines, in line with the
+existing practice of saving and restoring registers in batches to match the
+stack alignment (which saves on code size).
+
+For the `LP64`, `LP64F`, `LP64D`, and `LP64Q` ABIs, the save restore routines
+use `roundup(N+1, 2) * 8` bytes of stack space (where `roundup(val, multiple)`
+rounds `value` up to a multiple of `multiple`).
+
+For the `ILP32`, `ILP32F`, and `ILP32D` ABIs, the save restore routines use
+`roundup(N+1, 4) * 4` bytes of stack space.
+
+For the `ILP32E` ABI, the save restore routines use `(N+1) * 4` bytes of stack
+space (which reflects the lower stack alignment used by this ABI).
+
+In all the save restore routines, across all ABIs, `ra` is stored adjacent to
+the incoming stack pointer (highest address), then the Callee-Saved registers in
+register order from `s0` to `s11`. This follows the [Frame Pointer
+Convention](https://github.com/riscv-non-isa/riscv-elf-psabi-doc/blob/master/riscv-cc.adoc#frame-pointer-convention),
+whether or not a frame pointer is actually being used, and contradicts with the
+order used by Zcmp push/pop instructions.
+
 == Conventions for vendor extensions
 
 Support for custom instruction set extensions are an important part of RISC-V,