NFC: Infrastructure changes for DXIL op vector and multi-dim overloads (microsoft#7259)

This change adds vector and multi-dimensional overload support for DXIL
operations.

Multi-dimensional (or "extended") overloads are added, where two or more
types in a DXIL Op function signature may vary independently, such as
both the return type and a parameter type. Until now, only one overload
dimension has been necessary.

For single-dim overloads, any number of parameters in a DXIL op may
refer to this single overload type.
For multi-dim overloads, each type that can vary must have a unique
overload dimension, even when two or more types must be the same. This
follows a pattern from llvm intrinsics. If two or more of the types need
to be the same, this constraint must be handled manually, outside the
automatic overload constraints defined by the DXIL op definitions.

Vector overloads are also added, requiring an additional set of scalar
overload types to define the allowed vector element types, on top of the
original set describing the allowed scalar overloads for an operation,
since both scalar and vector overloads may be allowed on the same
operation.

There are several components involved in handling DXIL operation
overloads, with some changes:

- DXIL Op definitions in `hctdb.py` use a string of characters to define
the allowed overloads, and special type names used in parameter
definitions that refer to the overload type.
  - Overload string syntax updated and more heavily validated.
  - `','` may separate dimensions for multi-dim overloads
- `'<'` indicates that a vector overload is allowed, in which case,
scalar components on the left indicate normal scalar overloads allowed,
and scalar components on the right indicate the allowed vector element
overloads.
- If scalar overloads are present to the left, and omitted to the right,
the scalar components are replicated to the right automatically. For
instance: `"hf<"` is equivalent to `"hf<hf"`.
- `dxil_max_overload_dims = 2` is introduced to define the maximum
number of overload dimensions currently supported.
- This is used to generate the `DXIL::kDxilMaxOloadDims` definition in
`DxilConstants.h`.
- `"$x0"` and `"$x1"` are used to reference each overloaded dxil type in
parameter definitions when more than one overload dimension is defined
for a DXIL op. Other special overload types are not allowed for
multi-dim overloads, which means you cannot (currently) describe a
multi-dim overload where a returned overload type is wrapped in a
resource return struct along with residency status. This could be
changed in the future if necessary.
- Enforced rules for multi-dim overloads keep them compatible with the
llvm intrinsic overloading scheme.
- `hctdb_instrhelp.py` translates overload and param type info from DXIL
operation definitions into code inserted into `DxilOperations.cpp`.
- `DxilOperations.h|cpp` encodes allowed overloads inside
`OpCodeProperty` state for each operation in the `m_OpCodeProps` table.
It uses this information, along with generated code, to enforce overload
rules on DXIL ops.
- The allowed overload definition in `OpCodeProperty` has been rewritten
to use a more compact `OverloadMask` type, support multi-dim overloads,
and add a second layer `AllowedVectorElements` for vector overloads for
each dimension.
- There are assumptions that one `llvm::Type*` describes the overload
type, such as with: `GetOpFunc`, `GetOpFuncList`, `GetOverloadType`,
`IsOverloadLegal`, and `m_OpCodeClassCache`. The scheme used for
multi-dim overloads is to encode each of the overload types in a single
unnamed `StructType`, like `type {i32, <4 x float>}`. This makes it
compatible with all these existing mechanisms without requiring an API
overhaul impacting the broader code base. `GetExtendedOverloadType` is
used to construct this type from multiple types.

While updating `DxilOperations.h|cpp`, I noticed and removed some unused
methods: `IsDxilOpTypeName`, `IsDxilOpType`, `IsDupDxilOpType`,
`GetOriginalDxilOpType`.

---------

Co-authored-by: Greg Roth <[email protected]>

Author: tex3d

include/dxc/DXIL/DxilConstants.h

@@ -155,6 +155,11 @@ const float kMinMipLodBias = -16.0f;
 
 const unsigned kResRetStatusIndex = 4;
 
+/* <py::lines('OLOAD_DIMS-TEXT')>hctdb_instrhelp.get_max_oload_dims()</py>*/
+// OLOAD_DIMS-TEXT:BEGIN
+const unsigned kDxilMaxOloadDims = 2;
+// OLOAD_DIMS-TEXT:END
+
 enum class ComponentType : uint32_t {
   Invalid = 0,
   I1,

include/dxc/DXIL/DxilOperations.h

@@ -57,12 +57,31 @@ class OP {
   // caches.
   void RefreshCache();
 
+  // The single llvm::Type * "OverloadType" has one of these forms:
+  // No overloads (NumOverloadDims == 0):
+  //  - TS_Void: VoidTy
+  // For single overload dimension (NumOverloadDims == 1):
+  //  - TS_F*, TS_I*: a scalar numeric type (half, float, i1, i64, etc.),
+  //  - TS_UDT: a pointer to a StructType representing a User Defined Type,
+  //  - TS_Object: a named StructType representing a built-in object, or
+  //  - TS_Vector: a vector type (<4 x float>, <16 x i16>, etc.)
+  // For multiple overload dimensions (TS_Extended, NumOverloadDims > 1):
+  //  - an unnamed StructType containing each type for the corresponding
+  //    dimension, such as: type { i32, <2 x float> }
+  //  - contained type options are the same as for single dimension.
+
   llvm::Function *GetOpFunc(OpCode OpCode, llvm::Type *pOverloadType);
+
+  // N-dimension convenience version of GetOpFunc:
+  llvm::Function *GetOpFunc(OpCode OpCode,
+                            llvm::ArrayRef<llvm::Type *> OverloadTypes);
+
   const llvm::SmallMapVector<llvm::Type *, llvm::Function *, 8> &
   GetOpFuncList(OpCode OpCode) const;
   bool IsDxilOpUsed(OpCode opcode) const;
   void RemoveFunction(llvm::Function *F);
   llvm::LLVMContext &GetCtx() { return m_Ctx; }
+  llvm::Module *GetModule() { return m_pModule; }
   llvm::Type *GetHandleType() const;
   llvm::Type *GetHitObjectType() const;
   llvm::Type *GetNodeHandleType() const;
@@ -81,9 +100,14 @@ class OP {
 
   llvm::Type *GetResRetType(llvm::Type *pOverloadType);
   llvm::Type *GetCBufferRetType(llvm::Type *pOverloadType);
-  llvm::Type *GetVectorType(unsigned numElements, llvm::Type *pOverloadType);
+  llvm::Type *GetStructVectorType(unsigned numElements,
+                                  llvm::Type *pOverloadType);
   bool IsResRetType(llvm::Type *Ty);
 
+  // Construct an unnamed struct type containing the set of member types.
+  llvm::StructType *
+  GetExtendedOverloadType(llvm::ArrayRef<llvm::Type *> OverloadTypes);
+
   // Try to get the opcode class for a function.
   // Return true and set `opClass` if the given function is a dxil function.
   // Return false if the given function is not a dxil function.
@@ -128,11 +152,6 @@ class OP {
   static bool BarrierRequiresGroup(const llvm::CallInst *CI);
   static bool BarrierRequiresNode(const llvm::CallInst *CI);
   static DXIL::BarrierMode TranslateToBarrierMode(const llvm::CallInst *CI);
-  static bool IsDxilOpTypeName(llvm::StringRef name);
-  static bool IsDxilOpType(llvm::StructType *ST);
-  static bool IsDupDxilOpType(llvm::StructType *ST);
-  static llvm::StructType *GetOriginalDxilOpType(llvm::StructType *ST,
-                                                 llvm::Module &M);
   static void GetMinShaderModelAndMask(OpCode C, bool bWithTranslation,
                                        unsigned &major, unsigned &minor,
                                        unsigned &mask);
@@ -141,6 +160,13 @@ class OP {
                                        unsigned valMinor, unsigned &major,
                                        unsigned &minor, unsigned &mask);
 
+  static bool IsDxilOpExtendedOverload(OpCode C);
+
+  // Return true if the overload name suffix for this operation may be
+  // constructed based on a user-defined or user-influenced type name
+  // that may not represent the same type in different linked modules.
+  static bool MayHaveNonCanonicalOverload(OpCode OC);
+
 private:
   // Per-module properties.
   llvm::LLVMContext &m_Ctx;
@@ -164,13 +190,33 @@ class OP {
 
   DXIL::LowPrecisionMode m_LowPrecisionMode;
 
-  static const unsigned kUserDefineTypeSlot = 9;
-  static const unsigned kObjectTypeSlot = 10;
-  static const unsigned kNumTypeOverloads =
-      11; // void, h,f,d, i1, i8,i16,i32,i64, udt, obj
+  // Overload types are split into "basic" overload types and special types
+  // Basic: void, half, float, double, i1, i8, i16, i32, i64
+  //  - These have one canonical overload per TypeSlot
+  // Special: udt, obj, vec, extended
+  //  - These may have many overloads per type slot
+  enum TypeSlot : unsigned {
+    TS_F16 = 0,
+    TS_F32 = 1,
+    TS_F64 = 2,
+    TS_I1 = 3,
+    TS_I8 = 4,
+    TS_I16 = 5,
+    TS_I32 = 6,
+    TS_I64 = 7,
+    TS_BasicCount,
+    TS_UDT = 8,      // Ex: %"struct.MyStruct" *
+    TS_Object = 9,   // Ex: %"class.StructuredBuffer<Foo>"
+    TS_Vector = 10,  // Ex: <8 x i16>
+    TS_MaskBitCount, // Types used in Mask end here
+    // TS_Extended is only used to identify the unnamed struct type used to wrap
+    // multiple overloads when using GetTypeSlot.
+    TS_Extended, // Ex: type { float, <16 x i32> }
+    TS_Invalid = UINT_MAX,
+  };
 
-  llvm::Type *m_pResRetType[kNumTypeOverloads];
-  llvm::Type *m_pCBufferRetType[kNumTypeOverloads];
+  llvm::Type *m_pResRetType[TS_BasicCount];
+  llvm::Type *m_pCBufferRetType[TS_BasicCount];
 
   struct OpCodeCacheItem {
     llvm::SmallMapVector<llvm::Type *, llvm::Function *, 8> pOverloads;
@@ -181,27 +227,46 @@ class OP {
 
 private:
   // Static properties.
+  struct OverloadMask {
+    // mask of type slot bits as (1 << TypeSlot)
+    uint16_t SlotMask;
+    static_assert(TS_MaskBitCount <= (sizeof(SlotMask) * 8));
+    bool operator[](unsigned TypeSlot) const {
+      return (TypeSlot < TS_MaskBitCount) ? (bool)(SlotMask & (1 << TypeSlot))
+                                          : 0;
+    }
+    operator bool() const { return SlotMask != 0; }
+  };
   struct OpCodeProperty {
     OpCode opCode;
     const char *pOpCodeName;
     OpCodeClass opCodeClass;
     const char *pOpCodeClassName;
-    bool bAllowOverload[kNumTypeOverloads]; // void, h,f,d, i1, i8,i16,i32,i64,
-                                            // udt
     llvm::Attribute::AttrKind FuncAttr;
+
+    // Number of overload dimensions used by the operation.
+    unsigned int NumOverloadDims;
+
+    // Mask of supported overload types for each overload dimension.
+    OverloadMask AllowedOverloads[DXIL::kDxilMaxOloadDims];
+
+    // Mask of scalar components allowed for each demension where
+    // AllowedOverloads[n][TS_Vector] is true.
+    OverloadMask AllowedVectorElements[DXIL::kDxilMaxOloadDims];
   };
   static const OpCodeProperty m_OpCodeProps[(unsigned)OpCode::NumOpCodes];
 
-  static const char *m_OverloadTypeName[kNumTypeOverloads];
+  static const char *m_OverloadTypeName[TS_BasicCount];
   static const char *m_NamePrefix;
   static const char *m_TypePrefix;
   static const char *m_MatrixTypePrefix;
   static unsigned GetTypeSlot(llvm::Type *pType);
   static const char *GetOverloadTypeName(unsigned TypeSlot);
-  static llvm::StringRef GetTypeName(llvm::Type *Ty, std::string &str);
-  static llvm::StringRef ConstructOverloadName(llvm::Type *Ty,
-                                               DXIL::OpCode opCode,
-                                               std::string &funcNameStorage);
+  static llvm::StringRef GetTypeName(llvm::Type *Ty,
+                                     llvm::SmallVectorImpl<char> &Storage);
+  static llvm::StringRef
+  ConstructOverloadName(llvm::Type *Ty, DXIL::OpCode opCode,
+                        llvm::SmallVectorImpl<char> &Storage);
 };
 
 } // namespace hlsl