Skip to content

[NFC] Refactoring MCDXBC to support out of order storage of root parameters #137284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
May 16, 2025
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 69 additions & 6 deletions llvm/include/llvm/MC/DXContainerRootSignature.h
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,81 @@
#include "llvm/BinaryFormat/DXContainer.h"
#include <cstdint>
#include <limits>
#include <variant>

namespace llvm {

class raw_ostream;
namespace mcdxbc {

struct RootParameter {
struct RootParameterInfo {
dxbc::RootParameterHeader Header;
union {
dxbc::RootConstants Constants;
dxbc::RTS0::v2::RootDescriptor Descriptor;
};
size_t Location;

RootParameterInfo() = default;

RootParameterInfo(dxbc::RootParameterHeader H, size_t L)
: Header(H), Location(L) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clearer to call these Header and Location rather than abbreviating here

Suggested change
RootParameterInfo(dxbc::RootParameterHeader H, size_t L)
: Header(H), Location(L) {}
RootParameterInfo(dxbc::RootParameterHeader Header, size_t Location)
: Header(Header), Location(Location) {}

};

using RootDescriptor = std::variant<dxbc::RTS0::v1::RootDescriptor,
dxbc::RTS0::v2::RootDescriptor>;
using ParametersView = std::variant<const dxbc::RootConstants *,
const dxbc::RTS0::v1::RootDescriptor *,
const dxbc::RTS0::v2::RootDescriptor *>;
struct RootParametersContainer {
SmallVector<RootParameterInfo> ParametersInfo;

SmallVector<dxbc::RootConstants> Constants;
SmallVector<RootDescriptor> Descriptors;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're not tied to the binary format itself here, so I think it would be quite a bit simpler to just have this vector contain the v2 descriptors:

  SmallVector<dxbc::RTS0::v2::RootDescriptor> Descriptors;

This should end up simplifying RootSignatureDesc::write, as we have the version there, so we can just check the version for the later fields when we write it out:

      support::endian::write(BOS, Descriptor.ShaderRegister,
                             llvm::endianness::little);
      support::endian::write(BOS, Descriptor.RegisterSpace,
                             llvm::endianness::little);
      if (Version > 1)
        support::endian::write(BOS, Descriptor.Flags, llvm::endianness::little);


void addInfo(dxbc::RootParameterHeader H, size_t L) {
ParametersInfo.push_back(RootParameterInfo(H, L));
}

void addParameter(dxbc::RootParameterHeader H, dxbc::RootConstants C) {
addInfo(H, Constants.size());
Constants.push_back(C);
}

void addParameter(dxbc::RootParameterHeader H,
dxbc::RTS0::v1::RootDescriptor D) {
addInfo(H, Descriptors.size());
Descriptors.push_back(D);
}

void addParameter(dxbc::RootParameterHeader H,
dxbc::RTS0::v2::RootDescriptor D) {
addInfo(H, Descriptors.size());
Descriptors.push_back(D);
}

std::optional<ParametersView> getParameter(const RootParameterInfo *H) const {
switch (H->Header.ParameterType) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this switch exhaustive? If it is then do we need to make the return optional?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, is using an optional necessary here? Is the nullopt case reachable? If it should not be reachable likely best to use an llvm_unreachable instead.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional is necessary here, because yaml2obj needs to be able to write invalid root signatures representations, since we used it as a testing tool.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand, can you elaborate. What does writing an invalid root signature look like and how does it interact with this code?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Some context, Root Parameters are split into a Header and a Data section. The Header contains a RootParameter type field, specifying which kind of data is being stored: Root Constants, Root Descriptors or Descriptor Tables. The header also contains an offset, pointing to the exact location of the data in the binary file. Here is an test example showing what an invalid root signature look like: https://github.com/llvm/llvm-project/blob/038d357dde4907d39f6a3fabbaf48dc39cf9dc60/llvm/test/ObjectYAML/DXContainer/RootSignature-InvalidType.yaml.

Notice that in such test there is no data section, only the header section. Since I don't know what kind of data it is/the data is not supported, it is not possible to write it.

case llvm::to_underlying(dxbc::RootParameterType::Constants32Bit):
return &Constants[H->Location];
case llvm::to_underlying(dxbc::RootParameterType::CBV):
case llvm::to_underlying(dxbc::RootParameterType::SRV):
case llvm::to_underlying(dxbc::RootParameterType::UAV):
const RootDescriptor &VersionedParam = Descriptors[H->Location];
if (std::holds_alternative<dxbc::RTS0::v1::RootDescriptor>(
VersionedParam)) {
return &std::get<dxbc::RTS0::v1::RootDescriptor>(VersionedParam);
}
return &std::get<dxbc::RTS0::v2::RootDescriptor>(VersionedParam);
}

return std::nullopt;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not really convinced that the abstraction that the std::variant and the generic getParameter are providing are really helping much here. Consider this simplified API:

  const std::pair<uint32_t, uint32_t>
  getTypeAndLocForParameter(uint32_t Index) const {
    const RootParameterInfo &Info = ParametersInfo[Index];
    return {Info.Header.ParameterType, Info.Location};
  }

  const dxbc::RootConstants &getConstant(size_t Index) const {
    return Constants[Index];
  }

  const dxbc::RTS0::v2::RootDescriptor &getRootDescriptor(size_t Index) const {
    return Descriptors[Index];
  }

The logic to use this is more or less the same - instead of the holds_alternative type checks we simply check against the type enum we already have:

-    auto P = ParametersContainer.getParameter(ParametersContainer[I]);
-    if (std::holds_alternative<const dxbc::RootConstants *>(P.value())) {
-      auto *Constants = std::get<const dxbc::RootConstants *>(P.value());
-      support::endian::write(BOS, Constants->ShaderRegister,
-                             llvm::endianness::little);
+    const auto &[Type, Loc] = ParametersContainer.getTypeAndLocForParameter(I);
+    switch (Type) {
+    case llvm::to_underlying(dxbc::RootParameterType::Constants32Bit): {
+      const dxbc::RootConstants &Constants =
+          ParametersContainer.getConstant(Loc);
+      support::endian::write(BOS, Constants.ShaderRegister,
+                             llvm::endianness::little);

I think std::variant has its uses when we need something akin to a type safe union or we want to use visitor patterns to handle a large number of cases, but if we're just going to switch over the types anyway I think it just adds a layer of abstraction that needs to be looked through when reading the code to understand it.


size_t size() const { return ParametersInfo.size(); }

SmallVector<RootParameterInfo>::const_iterator begin() const {
return ParametersInfo.begin();
}
SmallVector<RootParameterInfo>::const_iterator end() const {
return ParametersInfo.end();
}
};
struct RootSignatureDesc {

Expand All @@ -29,7 +92,7 @@ struct RootSignatureDesc {
uint32_t RootParameterOffset = 0U;
uint32_t StaticSamplersOffset = 0u;
uint32_t NumStaticSamplers = 0u;
SmallVector<mcdxbc::RootParameter> Parameters;
mcdxbc::RootParametersContainer ParametersContainer;

void write(raw_ostream &OS) const;

Expand Down
2 changes: 1 addition & 1 deletion llvm/include/llvm/ObjectYAML/DXContainerYAML.h
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ struct RootParameterYamlDesc {
uint32_t Type;
uint32_t Visibility;
uint32_t Offset;
RootParameterYamlDesc() {};
RootParameterYamlDesc(){};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like an accidental reformat here.

RootParameterYamlDesc(uint32_t T) : Type(T) {
switch (T) {

Expand Down
75 changes: 40 additions & 35 deletions llvm/lib/MC/DXContainerRootSignature.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -30,24 +30,20 @@ static void rewriteOffsetToCurrentByte(raw_svector_ostream &Stream,

size_t RootSignatureDesc::getSize() const {
size_t Size = sizeof(dxbc::RootSignatureHeader) +
Parameters.size() * sizeof(dxbc::RootParameterHeader);
ParametersContainer.size() * sizeof(dxbc::RootParameterHeader);

for (const mcdxbc::RootParameter &P : Parameters) {
switch (P.Header.ParameterType) {
case llvm::to_underlying(dxbc::RootParameterType::Constants32Bit):
Size += sizeof(dxbc::RootConstants);
break;
case llvm::to_underlying(dxbc::RootParameterType::CBV):
case llvm::to_underlying(dxbc::RootParameterType::SRV):
case llvm::to_underlying(dxbc::RootParameterType::UAV):
if (Version == 1)
Size += sizeof(dxbc::RTS0::v1::RootDescriptor);
else
Size += sizeof(dxbc::RTS0::v2::RootDescriptor);

break;
}
for (const auto &I : ParametersContainer) {
std::optional<ParametersView> P = ParametersContainer.getParameter(&I);
if (!P)
continue;
std::visit(
[&Size](auto &Value) -> void {
using T = std::decay_t<decltype(*Value)>;
Size += sizeof(T);
},
*P);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you end up taking my suggestions for the simplified API above this will probably just revert to what it was before.


return Size;
}

Expand All @@ -56,7 +52,7 @@ void RootSignatureDesc::write(raw_ostream &OS) const {
raw_svector_ostream BOS(Storage);
BOS.reserveExtraSpace(getSize());

const uint32_t NumParameters = Parameters.size();
const uint32_t NumParameters = ParametersContainer.size();

support::endian::write(BOS, Version, llvm::endianness::little);
support::endian::write(BOS, NumParameters, llvm::endianness::little);
Expand All @@ -66,7 +62,7 @@ void RootSignatureDesc::write(raw_ostream &OS) const {
support::endian::write(BOS, Flags, llvm::endianness::little);

SmallVector<uint32_t> ParamsOffsets;
for (const mcdxbc::RootParameter &P : Parameters) {
for (const auto &P : ParametersContainer) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think auto is better than writing out the type here.

support::endian::write(BOS, P.Header.ParameterType,
llvm::endianness::little);
support::endian::write(BOS, P.Header.ShaderVisibility,
Expand All @@ -76,29 +72,38 @@ void RootSignatureDesc::write(raw_ostream &OS) const {
}

assert(NumParameters == ParamsOffsets.size());
for (size_t I = 0; I < NumParameters; ++I) {
const RootParameterInfo *H = ParametersContainer.begin();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

H isn't used any more (it's merely incremented)

for (size_t I = 0; I < NumParameters; ++I, H++) {
rewriteOffsetToCurrentByte(BOS, ParamsOffsets[I]);
const mcdxbc::RootParameter &P = Parameters[I];

switch (P.Header.ParameterType) {
case llvm::to_underlying(dxbc::RootParameterType::Constants32Bit):
support::endian::write(BOS, P.Constants.ShaderRegister,
auto P = ParametersContainer.getParameter(H);
if (!P)
continue;
if (std::holds_alternative<const dxbc::RootConstants *>(P.value())) {
auto *Constants = std::get<const dxbc::RootConstants *>(P.value());
support::endian::write(BOS, Constants->ShaderRegister,
llvm::endianness::little);
support::endian::write(BOS, P.Constants.RegisterSpace,
support::endian::write(BOS, Constants->RegisterSpace,
llvm::endianness::little);
support::endian::write(BOS, P.Constants.Num32BitValues,
support::endian::write(BOS, Constants->Num32BitValues,
llvm::endianness::little);
break;
case llvm::to_underlying(dxbc::RootParameterType::CBV):
case llvm::to_underlying(dxbc::RootParameterType::SRV):
case llvm::to_underlying(dxbc::RootParameterType::UAV):
support::endian::write(BOS, P.Descriptor.ShaderRegister,
} else if (std::holds_alternative<const dxbc::RTS0::v1::RootDescriptor *>(
*P)) {
auto *Descriptor =
std::get<const dxbc::RTS0::v1::RootDescriptor *>(P.value());
support::endian::write(BOS, Descriptor->ShaderRegister,
llvm::endianness::little);
support::endian::write(BOS, Descriptor->RegisterSpace,
llvm::endianness::little);
} else if (std::holds_alternative<const dxbc::RTS0::v2::RootDescriptor *>(
*P)) {
auto *Descriptor =
std::get<const dxbc::RTS0::v2::RootDescriptor *>(P.value());

support::endian::write(BOS, Descriptor->ShaderRegister,
llvm::endianness::little);
support::endian::write(BOS, P.Descriptor.RegisterSpace,
support::endian::write(BOS, Descriptor->RegisterSpace,
llvm::endianness::little);
if (Version > 1)
support::endian::write(BOS, P.Descriptor.Flags,
llvm::endianness::little);
support::endian::write(BOS, Descriptor->Flags, llvm::endianness::little);
}
}
assert(Storage.size() == getSize());
Expand Down
36 changes: 24 additions & 12 deletions llvm/lib/ObjectYAML/DXContainerEmitter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -274,27 +274,39 @@ void DXContainerWriter::writeParts(raw_ostream &OS) {
RS.StaticSamplersOffset = P.RootSignature->StaticSamplersOffset;

for (const auto &Param : P.RootSignature->Parameters) {
mcdxbc::RootParameter NewParam;
NewParam.Header = dxbc::RootParameterHeader{
Param.Type, Param.Visibility, Param.Offset};
auto Header = dxbc::RootParameterHeader{Param.Type, Param.Visibility,
Param.Offset};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
auto Header = dxbc::RootParameterHeader{Param.Type, Param.Visibility,
Param.Offset};
dxbc::RootParameterHeader Header{Param.Type, Param.Visibility,
Param.Offset};


switch (Param.Type) {
case llvm::to_underlying(dxbc::RootParameterType::Constants32Bit):
NewParam.Constants.Num32BitValues = Param.Constants.Num32BitValues;
NewParam.Constants.RegisterSpace = Param.Constants.RegisterSpace;
NewParam.Constants.ShaderRegister = Param.Constants.ShaderRegister;
dxbc::RootConstants Constants;
Constants.Num32BitValues = Param.Constants.Num32BitValues;
Constants.RegisterSpace = Param.Constants.RegisterSpace;
Constants.ShaderRegister = Param.Constants.ShaderRegister;
RS.ParametersContainer.addParameter(Header, Constants);
break;
case llvm::to_underlying(dxbc::RootParameterType::SRV):
case llvm::to_underlying(dxbc::RootParameterType::UAV):
case llvm::to_underlying(dxbc::RootParameterType::CBV):
NewParam.Descriptor.RegisterSpace = Param.Descriptor.RegisterSpace;
NewParam.Descriptor.ShaderRegister = Param.Descriptor.ShaderRegister;
if (P.RootSignature->Version > 1)
NewParam.Descriptor.Flags = Param.Descriptor.getEncodedFlags();
if (RS.Version == 1) {
dxbc::RTS0::v1::RootDescriptor Descriptor;
Descriptor.RegisterSpace = Param.Descriptor.RegisterSpace;
Descriptor.ShaderRegister = Param.Descriptor.ShaderRegister;
RS.ParametersContainer.addParameter(Header, Descriptor);
} else {
dxbc::RTS0::v2::RootDescriptor Descriptor;
Descriptor.RegisterSpace = Param.Descriptor.RegisterSpace;
Descriptor.ShaderRegister = Param.Descriptor.ShaderRegister;
Descriptor.Flags = Param.Descriptor.getEncodedFlags();
RS.ParametersContainer.addParameter(Header, Descriptor);
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems clearer than the implicit cast to from the v1 to the v2 descriptor:

          dxbc::RTS0::v2::RootDescriptor Descriptor;
          Descriptor.RegisterSpace = Param.Descriptor.RegisterSpace;
          Descriptor.ShaderRegister = Param.Descriptor.ShaderRegister;
          if (RS.Version > 1)
            Descriptor.Flags = Param.Descriptor.getEncodedFlags();
          RS.ParametersContainer.addParameter(Header, Descriptor);

aside: We should probably make the v2::RootDescriptor constructor that takes a v1::RootDescriptor explicit - this implicit cast being legal seems dangerous

break;
default:
// Handling invalid parameter type edge case. We intentionally let
// obj2yaml/yaml2obj parse and emit invalid dxcontainer data, in order
// for that to be used as a testing tool more effectively.
RS.ParametersContainer.addInfo(Header, -1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure, but do you think it's worth adding a addUnknownParameter or addInvalidParameter method instead of calling addInfo here directly for clarity?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thinks that is a good idea. It can make the code more consistent. And reduce a little of the confusion regarding this edge case.

}

RS.Parameters.push_back(NewParam);
}

RS.write(OS);
Expand Down
47 changes: 26 additions & 21 deletions llvm/lib/Target/DirectX/DXILRootSignature.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -75,31 +75,32 @@ static bool parseRootConstants(LLVMContext *Ctx, mcdxbc::RootSignatureDesc &RSD,
if (RootConstantNode->getNumOperands() != 5)
return reportError(Ctx, "Invalid format for RootConstants Element");

mcdxbc::RootParameter NewParameter;
NewParameter.Header.ParameterType =
dxbc::RootParameterHeader Header;
Header.ParameterType =
llvm::to_underlying(dxbc::RootParameterType::Constants32Bit);

if (std::optional<uint32_t> Val = extractMdIntValue(RootConstantNode, 1))
NewParameter.Header.ShaderVisibility = *Val;
Header.ShaderVisibility = *Val;
else
return reportError(Ctx, "Invalid value for ShaderVisibility");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Header.ParameterOffset is left uninitialized. Bug?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to initialized Header.ParameterOffset, since those will be calculated when writing, in RootSignatureDesc::write

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like a bug when the structure is created here but then only partially initialized. We should probably have a comment here, or even write a zero to it with a comment that it will be overwritten later.


dxbc::RootConstants Constants;
if (std::optional<uint32_t> Val = extractMdIntValue(RootConstantNode, 2))
NewParameter.Constants.ShaderRegister = *Val;
Constants.ShaderRegister = *Val;
else
return reportError(Ctx, "Invalid value for ShaderRegister");

if (std::optional<uint32_t> Val = extractMdIntValue(RootConstantNode, 3))
NewParameter.Constants.RegisterSpace = *Val;
Constants.RegisterSpace = *Val;
else
return reportError(Ctx, "Invalid value for RegisterSpace");

if (std::optional<uint32_t> Val = extractMdIntValue(RootConstantNode, 4))
NewParameter.Constants.Num32BitValues = *Val;
Constants.Num32BitValues = *Val;
else
return reportError(Ctx, "Invalid value for Num32BitValues");

RSD.Parameters.push_back(NewParameter);
RSD.ParametersContainer.addParameter(Header, Constants);

return false;
}
Expand Down Expand Up @@ -164,12 +165,12 @@ static bool validate(LLVMContext *Ctx, const mcdxbc::RootSignatureDesc &RSD) {
return reportValueError(Ctx, "RootFlags", RSD.Flags);
}

for (const mcdxbc::RootParameter &P : RSD.Parameters) {
if (!dxbc::isValidShaderVisibility(P.Header.ShaderVisibility))
for (const llvm::mcdxbc::RootParameterInfo &Info : RSD.ParametersContainer) {
if (!dxbc::isValidShaderVisibility(Info.Header.ShaderVisibility))
return reportValueError(Ctx, "ShaderVisibility",
P.Header.ShaderVisibility);
Info.Header.ShaderVisibility);

assert(dxbc::isValidParameterType(P.Header.ParameterType) &&
assert(dxbc::isValidParameterType(Info.Header.ParameterType) &&
"Invalid value for ParameterType");
}

Expand Down Expand Up @@ -287,22 +288,26 @@ PreservedAnalyses RootSignatureAnalysisPrinter::run(Module &M,
OS << indent(Space) << "Version: " << RS.Version << "\n";
OS << indent(Space) << "RootParametersOffset: " << RS.RootParameterOffset
<< "\n";
OS << indent(Space) << "NumParameters: " << RS.Parameters.size() << "\n";
OS << indent(Space) << "NumParameters: " << RS.ParametersContainer.size()
<< "\n";
Space++;
for (auto const &P : RS.Parameters) {
OS << indent(Space) << "- Parameter Type: " << P.Header.ParameterType
for (auto const &Info : RS.ParametersContainer) {
OS << indent(Space) << "- Parameter Type: " << Info.Header.ParameterType
<< "\n";
OS << indent(Space + 2)
<< "Shader Visibility: " << P.Header.ShaderVisibility << "\n";
switch (P.Header.ParameterType) {
case llvm::to_underlying(dxbc::RootParameterType::Constants32Bit):
<< "Shader Visibility: " << Info.Header.ShaderVisibility << "\n";
std::optional<mcdxbc::ParametersView> P =
RS.ParametersContainer.getParameter(&Info);
if (!P)
continue;
if (std::holds_alternative<const dxbc::RootConstants *>(*P)) {
auto *Constants = std::get<const dxbc::RootConstants *>(*P);
OS << indent(Space + 2)
<< "Register Space: " << P.Constants.RegisterSpace << "\n";
<< "Register Space: " << Constants->RegisterSpace << "\n";
OS << indent(Space + 2)
<< "Shader Register: " << P.Constants.ShaderRegister << "\n";
<< "Shader Register: " << Constants->ShaderRegister << "\n";
OS << indent(Space + 2)
<< "Num 32 Bit Values: " << P.Constants.Num32BitValues << "\n";
break;
<< "Num 32 Bit Values: " << Constants->Num32BitValues << "\n";
}
}
Space--;
Expand Down
Loading