asm: refactor VEX implementation (#10845)

* asm: refactor VEX implementation

This change adapts the current VEX logic implemented in #10754 to look
more like the current infrastructure. To do so:
- it factors out a `VexPrefix` from the current `VexInstruction`,
  allowing us to reuse existing logic for ModR/M and SIB bytes (these do
  are the same in both REX and VEX encodings)
- it uses compile-time constant bits like the `L` or `pp` fields
  directly, hopefully saving a bit on compile time
- all of this allows more consistent code generation

On the DSL side, this significantly changes how one defines VEX
instructions. Many helper methods are added and now documented. It also
makes use of a slightly unrelated change included here: I factored out
how any encoding can affect the ModR/M byte with `ModRmKind`. This
`enum` forces the DSL to pick only one of `/r` or `/digit` when defining
instructions.

* review: add `VexW`

Author: abrown

cranelift/assembler-x64/meta/src/dsl.rs

@@ -8,10 +8,11 @@ mod encoding;
 mod features;
 pub mod format;
 
+pub use encoding::{Encoding, ModRmKind, OpcodeMod};
 pub use encoding::{
-    Encoding, Group1Prefix, Group2Prefix, Group3Prefix, Group4Prefix, Opcodes, Prefixes, Rex, Vex,
-    VexLength, VexMMMMM, VexPP, rex, vex,
+    Group1Prefix, Group2Prefix, Group3Prefix, Group4Prefix, Opcodes, Prefixes, Rex, rex,
 };
+pub use encoding::{Vex, VexEscape, VexLength, VexPrefix, vex};
 pub use features::{ALL_FEATURES, Feature, Features};
 pub use format::{Extension, Format, Location, Mutability, Operand, OperandKind, RegClass};
 pub use format::{align, fmt, implicit, r, rw, sxl, sxq, sxw, w};